Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
895.975297ms
FAIL (2025-08-12 21:06:20 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 12 20:22:01.339537006 (98h55m42.070804636s) [upload=103006]: Starting upload
Aug 12 20:22:01.339603582 (98h55m42.070871202s) [upload=103006]: Done starting upload
Aug 12 20:22:01.339679935 (98h55m42.070947565s) [upload=103006]: Uploading segments...
Aug 12 20:22:01.339853221 (98h55m42.071120851s) [upload=103006]: Got next segment. Inline: false
Aug 12 20:22:01.339892445 (98h55m42.071160075s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 12 20:22:01.339896492 (98h55m42.071164122s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 12 20:22:01.379548451 (98h55m42.110816081s) [seg_pos={0 0}, upload=103006]: Begin upload segment...
Aug 12 20:22:01.379584901 (98h55m42.110852531s) [seg_pos={0 0}, upload=103006]: Done begin upload segment.
Aug 12 20:22:01.379611641 (98h55m42.110879271s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.379628362 (98h55m42.110895992s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380012595 (98h55m42.111280215s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380023856 (98h55m42.111291476s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380277303 (98h55m42.111544923s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380294425 (98h55m42.111562045s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380557831 (98h55m42.111825482s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381005865 (98h55m42.112273495s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381223243 (98h55m42.112490873s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381495867 (98h55m42.112763487s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381860323 (98h55m42.113127953s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382135862 (98h55m42.113403482s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382450184 (98h55m42.113717804s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382711605 (98h55m42.113979225s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382963689 (98h55m42.11423131s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.383222817 (98h55m42.114490438s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384120417 (98h55m42.115388047s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384281039 (98h55m42.115548659s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.503410855 (98h55m42.234678495s) [upload=103006]: Next returned nil segment
Aug 12 20:22:01.503425643 (98h55m42.234693273s) [upload=103006]: Waiting for error group managing segments...
Aug 12 20:22:01.518591716 (98h55m42.249859346s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.522295799 (98h55m42.253563429s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.524942810 (98h55m42.25621044s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529281676 (98h55m42.260549297s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529534613 (98h55m42.260802233s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529663065 (98h55m42.260930685s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.532249323 (98h55m42.263516953s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.537316489 (98h55m42.268584109s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.540779567 (98h55m42.272047197s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.550597055 (98h55m42.281864685s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.552054648 (98h55m42.283322299s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.557838805 (98h55m42.289106454s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.560393523 (98h55m42.291661173s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562560110 (98h55m42.29382776s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562600397 (98h55m42.293868037s) [seg_pos={0 0}, upload=103006]: Segment reached optimal threshold of 14 pieces.
Aug 12 20:22:01.562929186 (98h55m42.294196835s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.562988978 (98h55m42.294256627s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HSvQVuCNaPdj84e4ybE8YWZ77ff2gbhnjd4z9HTf1abGxz2u7): protocol: read tcp 5.161.89.252:22078->79.127.205.235:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563045493 (98h55m42.294313144s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17z3AiUfbxBX2eMDnUnqo1bbgojuQxJro7PsEaPjYzQjHQ3D6K): protocol: read tcp 5.161.89.252:35864->64.49.13.169:4713: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563079208 (98h55m42.294346858s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.563135343 (98h55m42.294402994s) [seg_pos={0 0}, upload=103006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 12 20:22:01.563200666 (98h55m42.294468316s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 12 20:22:01.563205484 (98h55m42.294473135s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 12 20:22:02.476106688 (98h55m43.207374368s) [upload=103006]: Done uploading segments...
Aug 12 20:22:02.476119001 (98h55m43.207386621s) [upload=103006]: Upload finished. err: <nil>
Duration: 896.002979ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [............X............................X..................] duration:
181.213554ms
FAIL (2025-08-12 21:06:07 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 20:51:52.367293922 (99h25m33.098561542s) [upload=103522]: Starting upload Aug 12 20:51:52.367341782 (99h25m33.098609412s) [upload=103522]: Done starting upload Aug 12 20:51:52.367390824 (99h25m33.098658444s) [upload=103522]: Uploading segments... Aug 12 20:51:52.367420861 (99h25m33.098688482s) [upload=103522]: Got next segment. Inline: true Aug 12 20:51:52.367424648 (99h25m33.098692278s) [upload=103522]: Waiting for error group managing segments... Aug 12 20:51:52.367460826 (99h25m33.098728456s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 20:51:52.367464623 (99h25m33.098732244s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 20:51:52.367467188 (99h25m33.098734818s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 20:51:52.369981799 (99h25m33.10124942s) [upload=103522]: Done uploading segments... Aug 12 20:51:52.369988672 (99h25m33.101256292s) [upload=103522]: Upload finished. err: metaclient: Too Many Requests
Duration: 181.233851ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
455.605282ms
FAIL (2025-08-12 21:06:01 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 12 20:22:01.339537006 (98h55m42.070804636s) [upload=103006]: Starting upload
Aug 12 20:22:01.339603582 (98h55m42.070871202s) [upload=103006]: Done starting upload
Aug 12 20:22:01.339679935 (98h55m42.070947565s) [upload=103006]: Uploading segments...
Aug 12 20:22:01.339853221 (98h55m42.071120851s) [upload=103006]: Got next segment. Inline: false
Aug 12 20:22:01.339892445 (98h55m42.071160075s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 12 20:22:01.339896492 (98h55m42.071164122s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 12 20:22:01.379548451 (98h55m42.110816081s) [seg_pos={0 0}, upload=103006]: Begin upload segment...
Aug 12 20:22:01.379584901 (98h55m42.110852531s) [seg_pos={0 0}, upload=103006]: Done begin upload segment.
Aug 12 20:22:01.379611641 (98h55m42.110879271s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.379628362 (98h55m42.110895992s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380012595 (98h55m42.111280215s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380023856 (98h55m42.111291476s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380277303 (98h55m42.111544923s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380294425 (98h55m42.111562045s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380557831 (98h55m42.111825482s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381005865 (98h55m42.112273495s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381223243 (98h55m42.112490873s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381495867 (98h55m42.112763487s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381860323 (98h55m42.113127953s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382135862 (98h55m42.113403482s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382450184 (98h55m42.113717804s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382711605 (98h55m42.113979225s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382963689 (98h55m42.11423131s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.383222817 (98h55m42.114490438s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384120417 (98h55m42.115388047s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384281039 (98h55m42.115548659s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.503410855 (98h55m42.234678495s) [upload=103006]: Next returned nil segment
Aug 12 20:22:01.503425643 (98h55m42.234693273s) [upload=103006]: Waiting for error group managing segments...
Aug 12 20:22:01.518591716 (98h55m42.249859346s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.522295799 (98h55m42.253563429s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.524942810 (98h55m42.25621044s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529281676 (98h55m42.260549297s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529534613 (98h55m42.260802233s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529663065 (98h55m42.260930685s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.532249323 (98h55m42.263516953s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.537316489 (98h55m42.268584109s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.540779567 (98h55m42.272047197s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.550597055 (98h55m42.281864685s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.552054648 (98h55m42.283322299s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.557838805 (98h55m42.289106454s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.560393523 (98h55m42.291661173s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562560110 (98h55m42.29382776s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562600397 (98h55m42.293868037s) [seg_pos={0 0}, upload=103006]: Segment reached optimal threshold of 14 pieces.
Aug 12 20:22:01.562929186 (98h55m42.294196835s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.562988978 (98h55m42.294256627s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HSvQVuCNaPdj84e4ybE8YWZ77ff2gbhnjd4z9HTf1abGxz2u7): protocol: read tcp 5.161.89.252:22078->79.127.205.235:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563045493 (98h55m42.294313144s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17z3AiUfbxBX2eMDnUnqo1bbgojuQxJro7PsEaPjYzQjHQ3D6K): protocol: read tcp 5.161.89.252:35864->64.49.13.169:4713: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563079208 (98h55m42.294346858s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.563135343 (98h55m42.294402994s) [seg_pos={0 0}, upload=103006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 12 20:22:01.563200666 (98h55m42.294468316s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 12 20:22:01.563205484 (98h55m42.294473135s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 12 20:22:02.476106688 (98h55m43.207374368s) [upload=103006]: Done uploading segments...
Aug 12 20:22:02.476119001 (98h55m43.207386621s) [upload=103006]: Upload finished. err: <nil>
Duration: 455.626933ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
761.224283ms
FAIL (2025-08-12 21:05:35 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 12 20:22:01.339537006 (98h55m42.070804636s) [upload=103006]: Starting upload
Aug 12 20:22:01.339603582 (98h55m42.070871202s) [upload=103006]: Done starting upload
Aug 12 20:22:01.339679935 (98h55m42.070947565s) [upload=103006]: Uploading segments...
Aug 12 20:22:01.339853221 (98h55m42.071120851s) [upload=103006]: Got next segment. Inline: false
Aug 12 20:22:01.339892445 (98h55m42.071160075s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 12 20:22:01.339896492 (98h55m42.071164122s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 12 20:22:01.379548451 (98h55m42.110816081s) [seg_pos={0 0}, upload=103006]: Begin upload segment...
Aug 12 20:22:01.379584901 (98h55m42.110852531s) [seg_pos={0 0}, upload=103006]: Done begin upload segment.
Aug 12 20:22:01.379611641 (98h55m42.110879271s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.379628362 (98h55m42.110895992s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380012595 (98h55m42.111280215s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380023856 (98h55m42.111291476s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380277303 (98h55m42.111544923s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380294425 (98h55m42.111562045s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380557831 (98h55m42.111825482s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381005865 (98h55m42.112273495s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381223243 (98h55m42.112490873s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381495867 (98h55m42.112763487s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381860323 (98h55m42.113127953s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382135862 (98h55m42.113403482s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382450184 (98h55m42.113717804s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382711605 (98h55m42.113979225s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382963689 (98h55m42.11423131s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.383222817 (98h55m42.114490438s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384120417 (98h55m42.115388047s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384281039 (98h55m42.115548659s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.503410855 (98h55m42.234678495s) [upload=103006]: Next returned nil segment
Aug 12 20:22:01.503425643 (98h55m42.234693273s) [upload=103006]: Waiting for error group managing segments...
Aug 12 20:22:01.518591716 (98h55m42.249859346s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.522295799 (98h55m42.253563429s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.524942810 (98h55m42.25621044s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529281676 (98h55m42.260549297s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529534613 (98h55m42.260802233s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529663065 (98h55m42.260930685s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.532249323 (98h55m42.263516953s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.537316489 (98h55m42.268584109s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.540779567 (98h55m42.272047197s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.550597055 (98h55m42.281864685s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.552054648 (98h55m42.283322299s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.557838805 (98h55m42.289106454s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.560393523 (98h55m42.291661173s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562560110 (98h55m42.29382776s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562600397 (98h55m42.293868037s) [seg_pos={0 0}, upload=103006]: Segment reached optimal threshold of 14 pieces.
Aug 12 20:22:01.562929186 (98h55m42.294196835s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.562988978 (98h55m42.294256627s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HSvQVuCNaPdj84e4ybE8YWZ77ff2gbhnjd4z9HTf1abGxz2u7): protocol: read tcp 5.161.89.252:22078->79.127.205.235:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563045493 (98h55m42.294313144s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17z3AiUfbxBX2eMDnUnqo1bbgojuQxJro7PsEaPjYzQjHQ3D6K): protocol: read tcp 5.161.89.252:35864->64.49.13.169:4713: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563079208 (98h55m42.294346858s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.563135343 (98h55m42.294402994s) [seg_pos={0 0}, upload=103006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 12 20:22:01.563200666 (98h55m42.294468316s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 12 20:22:01.563205484 (98h55m42.294473135s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 12 20:22:02.476106688 (98h55m43.207374368s) [upload=103006]: Done uploading segments...
Aug 12 20:22:02.476119001 (98h55m43.207386621s) [upload=103006]: Upload finished. err: <nil>
Duration: 761.261141ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.............X............................X.................] duration:
142.249311ms
FAIL (2025-08-12 21:05:09 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 20:51:52.367293922 (99h25m33.098561542s) [upload=103522]: Starting upload Aug 12 20:51:52.367341782 (99h25m33.098609412s) [upload=103522]: Done starting upload Aug 12 20:51:52.367390824 (99h25m33.098658444s) [upload=103522]: Uploading segments... Aug 12 20:51:52.367420861 (99h25m33.098688482s) [upload=103522]: Got next segment. Inline: true Aug 12 20:51:52.367424648 (99h25m33.098692278s) [upload=103522]: Waiting for error group managing segments... Aug 12 20:51:52.367460826 (99h25m33.098728456s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 20:51:52.367464623 (99h25m33.098732244s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 20:51:52.367467188 (99h25m33.098734818s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 20:51:52.369981799 (99h25m33.10124942s) [upload=103522]: Done uploading segments... Aug 12 20:51:52.369988672 (99h25m33.101256292s) [upload=103522]: Upload finished. err: metaclient: Too Many Requests
Duration: 142.277984ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
778.676681ms
FAIL (2025-08-12 21:04:52 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 12 20:22:01.339537006 (98h55m42.070804636s) [upload=103006]: Starting upload
Aug 12 20:22:01.339603582 (98h55m42.070871202s) [upload=103006]: Done starting upload
Aug 12 20:22:01.339679935 (98h55m42.070947565s) [upload=103006]: Uploading segments...
Aug 12 20:22:01.339853221 (98h55m42.071120851s) [upload=103006]: Got next segment. Inline: false
Aug 12 20:22:01.339892445 (98h55m42.071160075s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 12 20:22:01.339896492 (98h55m42.071164122s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 12 20:22:01.379548451 (98h55m42.110816081s) [seg_pos={0 0}, upload=103006]: Begin upload segment...
Aug 12 20:22:01.379584901 (98h55m42.110852531s) [seg_pos={0 0}, upload=103006]: Done begin upload segment.
Aug 12 20:22:01.379611641 (98h55m42.110879271s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.379628362 (98h55m42.110895992s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380012595 (98h55m42.111280215s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380023856 (98h55m42.111291476s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380277303 (98h55m42.111544923s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380294425 (98h55m42.111562045s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380557831 (98h55m42.111825482s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381005865 (98h55m42.112273495s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381223243 (98h55m42.112490873s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381495867 (98h55m42.112763487s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381860323 (98h55m42.113127953s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382135862 (98h55m42.113403482s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382450184 (98h55m42.113717804s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382711605 (98h55m42.113979225s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382963689 (98h55m42.11423131s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.383222817 (98h55m42.114490438s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384120417 (98h55m42.115388047s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384281039 (98h55m42.115548659s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.503410855 (98h55m42.234678495s) [upload=103006]: Next returned nil segment
Aug 12 20:22:01.503425643 (98h55m42.234693273s) [upload=103006]: Waiting for error group managing segments...
Aug 12 20:22:01.518591716 (98h55m42.249859346s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.522295799 (98h55m42.253563429s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.524942810 (98h55m42.25621044s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529281676 (98h55m42.260549297s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529534613 (98h55m42.260802233s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529663065 (98h55m42.260930685s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.532249323 (98h55m42.263516953s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.537316489 (98h55m42.268584109s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.540779567 (98h55m42.272047197s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.550597055 (98h55m42.281864685s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.552054648 (98h55m42.283322299s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.557838805 (98h55m42.289106454s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.560393523 (98h55m42.291661173s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562560110 (98h55m42.29382776s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562600397 (98h55m42.293868037s) [seg_pos={0 0}, upload=103006]: Segment reached optimal threshold of 14 pieces.
Aug 12 20:22:01.562929186 (98h55m42.294196835s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.562988978 (98h55m42.294256627s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HSvQVuCNaPdj84e4ybE8YWZ77ff2gbhnjd4z9HTf1abGxz2u7): protocol: read tcp 5.161.89.252:22078->79.127.205.235:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563045493 (98h55m42.294313144s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17z3AiUfbxBX2eMDnUnqo1bbgojuQxJro7PsEaPjYzQjHQ3D6K): protocol: read tcp 5.161.89.252:35864->64.49.13.169:4713: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563079208 (98h55m42.294346858s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.563135343 (98h55m42.294402994s) [seg_pos={0 0}, upload=103006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 12 20:22:01.563200666 (98h55m42.294468316s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 12 20:22:01.563205484 (98h55m42.294473135s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 12 20:22:02.476106688 (98h55m43.207374368s) [upload=103006]: Done uploading segments...
Aug 12 20:22:02.476119001 (98h55m43.207386621s) [upload=103006]: Upload finished. err: <nil>
Duration: 778.701658ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..............X............................X................] duration:
1.067308289s
FAIL (2025-08-12 21:04:08 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 20:51:52.367293922 (99h25m33.098561542s) [upload=103522]: Starting upload Aug 12 20:51:52.367341782 (99h25m33.098609412s) [upload=103522]: Done starting upload Aug 12 20:51:52.367390824 (99h25m33.098658444s) [upload=103522]: Uploading segments... Aug 12 20:51:52.367420861 (99h25m33.098688482s) [upload=103522]: Got next segment. Inline: true Aug 12 20:51:52.367424648 (99h25m33.098692278s) [upload=103522]: Waiting for error group managing segments... Aug 12 20:51:52.367460826 (99h25m33.098728456s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 20:51:52.367464623 (99h25m33.098732244s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 20:51:52.367467188 (99h25m33.098734818s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 20:51:52.369981799 (99h25m33.10124942s) [upload=103522]: Done uploading segments... Aug 12 20:51:52.369988672 (99h25m33.101256292s) [upload=103522]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.067322886s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
1.027997635s
FAIL (2025-08-12 21:03:59 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 12 20:22:01.339537006 (98h55m42.070804636s) [upload=103006]: Starting upload
Aug 12 20:22:01.339603582 (98h55m42.070871202s) [upload=103006]: Done starting upload
Aug 12 20:22:01.339679935 (98h55m42.070947565s) [upload=103006]: Uploading segments...
Aug 12 20:22:01.339853221 (98h55m42.071120851s) [upload=103006]: Got next segment. Inline: false
Aug 12 20:22:01.339892445 (98h55m42.071160075s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 12 20:22:01.339896492 (98h55m42.071164122s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 12 20:22:01.379548451 (98h55m42.110816081s) [seg_pos={0 0}, upload=103006]: Begin upload segment...
Aug 12 20:22:01.379584901 (98h55m42.110852531s) [seg_pos={0 0}, upload=103006]: Done begin upload segment.
Aug 12 20:22:01.379611641 (98h55m42.110879271s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.379628362 (98h55m42.110895992s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380012595 (98h55m42.111280215s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380023856 (98h55m42.111291476s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380277303 (98h55m42.111544923s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380294425 (98h55m42.111562045s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380557831 (98h55m42.111825482s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381005865 (98h55m42.112273495s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381223243 (98h55m42.112490873s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381495867 (98h55m42.112763487s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381860323 (98h55m42.113127953s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382135862 (98h55m42.113403482s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382450184 (98h55m42.113717804s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382711605 (98h55m42.113979225s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382963689 (98h55m42.11423131s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.383222817 (98h55m42.114490438s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384120417 (98h55m42.115388047s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384281039 (98h55m42.115548659s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.503410855 (98h55m42.234678495s) [upload=103006]: Next returned nil segment
Aug 12 20:22:01.503425643 (98h55m42.234693273s) [upload=103006]: Waiting for error group managing segments...
Aug 12 20:22:01.518591716 (98h55m42.249859346s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.522295799 (98h55m42.253563429s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.524942810 (98h55m42.25621044s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529281676 (98h55m42.260549297s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529534613 (98h55m42.260802233s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529663065 (98h55m42.260930685s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.532249323 (98h55m42.263516953s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.537316489 (98h55m42.268584109s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.540779567 (98h55m42.272047197s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.550597055 (98h55m42.281864685s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.552054648 (98h55m42.283322299s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.557838805 (98h55m42.289106454s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.560393523 (98h55m42.291661173s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562560110 (98h55m42.29382776s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562600397 (98h55m42.293868037s) [seg_pos={0 0}, upload=103006]: Segment reached optimal threshold of 14 pieces.
Aug 12 20:22:01.562929186 (98h55m42.294196835s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.562988978 (98h55m42.294256627s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HSvQVuCNaPdj84e4ybE8YWZ77ff2gbhnjd4z9HTf1abGxz2u7): protocol: read tcp 5.161.89.252:22078->79.127.205.235:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563045493 (98h55m42.294313144s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17z3AiUfbxBX2eMDnUnqo1bbgojuQxJro7PsEaPjYzQjHQ3D6K): protocol: read tcp 5.161.89.252:35864->64.49.13.169:4713: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563079208 (98h55m42.294346858s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.563135343 (98h55m42.294402994s) [seg_pos={0 0}, upload=103006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 12 20:22:01.563200666 (98h55m42.294468316s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 12 20:22:01.563205484 (98h55m42.294473135s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 12 20:22:02.476106688 (98h55m43.207374368s) [upload=103006]: Done uploading segments...
Aug 12 20:22:02.476119001 (98h55m43.207386621s) [upload=103006]: Upload finished. err: <nil>
Duration: 1.028022522s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...............X............................X...............] duration:
148.361286ms
FAIL (2025-08-12 21:03:22 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 20:51:52.367293922 (99h25m33.098561542s) [upload=103522]: Starting upload Aug 12 20:51:52.367341782 (99h25m33.098609412s) [upload=103522]: Done starting upload Aug 12 20:51:52.367390824 (99h25m33.098658444s) [upload=103522]: Uploading segments... Aug 12 20:51:52.367420861 (99h25m33.098688482s) [upload=103522]: Got next segment. Inline: true Aug 12 20:51:52.367424648 (99h25m33.098692278s) [upload=103522]: Waiting for error group managing segments... Aug 12 20:51:52.367460826 (99h25m33.098728456s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 20:51:52.367464623 (99h25m33.098732244s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 20:51:52.367467188 (99h25m33.098734818s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 20:51:52.369981799 (99h25m33.10124942s) [upload=103522]: Done uploading segments... Aug 12 20:51:52.369988672 (99h25m33.101256292s) [upload=103522]: Upload finished. err: metaclient: Too Many Requests
Duration: 148.39536ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
495.223373ms
FAIL (2025-08-12 21:03:12 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 12 20:22:01.339537006 (98h55m42.070804636s) [upload=103006]: Starting upload
Aug 12 20:22:01.339603582 (98h55m42.070871202s) [upload=103006]: Done starting upload
Aug 12 20:22:01.339679935 (98h55m42.070947565s) [upload=103006]: Uploading segments...
Aug 12 20:22:01.339853221 (98h55m42.071120851s) [upload=103006]: Got next segment. Inline: false
Aug 12 20:22:01.339892445 (98h55m42.071160075s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 12 20:22:01.339896492 (98h55m42.071164122s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 12 20:22:01.379548451 (98h55m42.110816081s) [seg_pos={0 0}, upload=103006]: Begin upload segment...
Aug 12 20:22:01.379584901 (98h55m42.110852531s) [seg_pos={0 0}, upload=103006]: Done begin upload segment.
Aug 12 20:22:01.379611641 (98h55m42.110879271s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.379628362 (98h55m42.110895992s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380012595 (98h55m42.111280215s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380023856 (98h55m42.111291476s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380277303 (98h55m42.111544923s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380294425 (98h55m42.111562045s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380557831 (98h55m42.111825482s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381005865 (98h55m42.112273495s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381223243 (98h55m42.112490873s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381495867 (98h55m42.112763487s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381860323 (98h55m42.113127953s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382135862 (98h55m42.113403482s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382450184 (98h55m42.113717804s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382711605 (98h55m42.113979225s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382963689 (98h55m42.11423131s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.383222817 (98h55m42.114490438s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384120417 (98h55m42.115388047s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384281039 (98h55m42.115548659s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.503410855 (98h55m42.234678495s) [upload=103006]: Next returned nil segment
Aug 12 20:22:01.503425643 (98h55m42.234693273s) [upload=103006]: Waiting for error group managing segments...
Aug 12 20:22:01.518591716 (98h55m42.249859346s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.522295799 (98h55m42.253563429s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.524942810 (98h55m42.25621044s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529281676 (98h55m42.260549297s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529534613 (98h55m42.260802233s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529663065 (98h55m42.260930685s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.532249323 (98h55m42.263516953s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.537316489 (98h55m42.268584109s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.540779567 (98h55m42.272047197s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.550597055 (98h55m42.281864685s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.552054648 (98h55m42.283322299s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.557838805 (98h55m42.289106454s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.560393523 (98h55m42.291661173s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562560110 (98h55m42.29382776s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562600397 (98h55m42.293868037s) [seg_pos={0 0}, upload=103006]: Segment reached optimal threshold of 14 pieces.
Aug 12 20:22:01.562929186 (98h55m42.294196835s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.562988978 (98h55m42.294256627s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HSvQVuCNaPdj84e4ybE8YWZ77ff2gbhnjd4z9HTf1abGxz2u7): protocol: read tcp 5.161.89.252:22078->79.127.205.235:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563045493 (98h55m42.294313144s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17z3AiUfbxBX2eMDnUnqo1bbgojuQxJro7PsEaPjYzQjHQ3D6K): protocol: read tcp 5.161.89.252:35864->64.49.13.169:4713: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563079208 (98h55m42.294346858s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.563135343 (98h55m42.294402994s) [seg_pos={0 0}, upload=103006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 12 20:22:01.563200666 (98h55m42.294468316s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 12 20:22:01.563205484 (98h55m42.294473135s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 12 20:22:02.476106688 (98h55m43.207374368s) [upload=103006]: Done uploading segments...
Aug 12 20:22:02.476119001 (98h55m43.207386621s) [upload=103006]: Upload finished. err: <nil>
Duration: 495.246056ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [................X............................X..............] duration:
196.900412ms
FAIL (2025-08-12 21:02:18 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 20:51:52.367293922 (99h25m33.098561542s) [upload=103522]: Starting upload Aug 12 20:51:52.367341782 (99h25m33.098609412s) [upload=103522]: Done starting upload Aug 12 20:51:52.367390824 (99h25m33.098658444s) [upload=103522]: Uploading segments... Aug 12 20:51:52.367420861 (99h25m33.098688482s) [upload=103522]: Got next segment. Inline: true Aug 12 20:51:52.367424648 (99h25m33.098692278s) [upload=103522]: Waiting for error group managing segments... Aug 12 20:51:52.367460826 (99h25m33.098728456s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 20:51:52.367464623 (99h25m33.098732244s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 20:51:52.367467188 (99h25m33.098734818s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 20:51:52.369981799 (99h25m33.10124942s) [upload=103522]: Done uploading segments... Aug 12 20:51:52.369988672 (99h25m33.101256292s) [upload=103522]: Upload finished. err: metaclient: Too Many Requests
Duration: 196.929948ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
755.27677ms
FAIL (2025-08-12 21:02:15 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 12 20:22:01.339537006 (98h55m42.070804636s) [upload=103006]: Starting upload
Aug 12 20:22:01.339603582 (98h55m42.070871202s) [upload=103006]: Done starting upload
Aug 12 20:22:01.339679935 (98h55m42.070947565s) [upload=103006]: Uploading segments...
Aug 12 20:22:01.339853221 (98h55m42.071120851s) [upload=103006]: Got next segment. Inline: false
Aug 12 20:22:01.339892445 (98h55m42.071160075s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 12 20:22:01.339896492 (98h55m42.071164122s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 12 20:22:01.379548451 (98h55m42.110816081s) [seg_pos={0 0}, upload=103006]: Begin upload segment...
Aug 12 20:22:01.379584901 (98h55m42.110852531s) [seg_pos={0 0}, upload=103006]: Done begin upload segment.
Aug 12 20:22:01.379611641 (98h55m42.110879271s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.379628362 (98h55m42.110895992s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380012595 (98h55m42.111280215s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380023856 (98h55m42.111291476s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380277303 (98h55m42.111544923s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380294425 (98h55m42.111562045s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380557831 (98h55m42.111825482s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381005865 (98h55m42.112273495s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381223243 (98h55m42.112490873s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381495867 (98h55m42.112763487s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381860323 (98h55m42.113127953s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382135862 (98h55m42.113403482s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382450184 (98h55m42.113717804s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382711605 (98h55m42.113979225s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382963689 (98h55m42.11423131s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.383222817 (98h55m42.114490438s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384120417 (98h55m42.115388047s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384281039 (98h55m42.115548659s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.503410855 (98h55m42.234678495s) [upload=103006]: Next returned nil segment
Aug 12 20:22:01.503425643 (98h55m42.234693273s) [upload=103006]: Waiting for error group managing segments...
Aug 12 20:22:01.518591716 (98h55m42.249859346s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.522295799 (98h55m42.253563429s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.524942810 (98h55m42.25621044s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529281676 (98h55m42.260549297s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529534613 (98h55m42.260802233s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529663065 (98h55m42.260930685s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.532249323 (98h55m42.263516953s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.537316489 (98h55m42.268584109s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.540779567 (98h55m42.272047197s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.550597055 (98h55m42.281864685s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.552054648 (98h55m42.283322299s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.557838805 (98h55m42.289106454s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.560393523 (98h55m42.291661173s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562560110 (98h55m42.29382776s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562600397 (98h55m42.293868037s) [seg_pos={0 0}, upload=103006]: Segment reached optimal threshold of 14 pieces.
Aug 12 20:22:01.562929186 (98h55m42.294196835s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.562988978 (98h55m42.294256627s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HSvQVuCNaPdj84e4ybE8YWZ77ff2gbhnjd4z9HTf1abGxz2u7): protocol: read tcp 5.161.89.252:22078->79.127.205.235:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563045493 (98h55m42.294313144s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17z3AiUfbxBX2eMDnUnqo1bbgojuQxJro7PsEaPjYzQjHQ3D6K): protocol: read tcp 5.161.89.252:35864->64.49.13.169:4713: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563079208 (98h55m42.294346858s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.563135343 (98h55m42.294402994s) [seg_pos={0 0}, upload=103006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 12 20:22:01.563200666 (98h55m42.294468316s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 12 20:22:01.563205484 (98h55m42.294473135s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 12 20:22:02.476106688 (98h55m43.207374368s) [upload=103006]: Done uploading segments...
Aug 12 20:22:02.476119001 (98h55m43.207386621s) [upload=103006]: Upload finished. err: <nil>
Duration: 755.307829ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.................X............................X.............] duration:
171.805519ms
FAIL (2025-08-12 21:01:26 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 20:51:52.367293922 (99h25m33.098561542s) [upload=103522]: Starting upload Aug 12 20:51:52.367341782 (99h25m33.098609412s) [upload=103522]: Done starting upload Aug 12 20:51:52.367390824 (99h25m33.098658444s) [upload=103522]: Uploading segments... Aug 12 20:51:52.367420861 (99h25m33.098688482s) [upload=103522]: Got next segment. Inline: true Aug 12 20:51:52.367424648 (99h25m33.098692278s) [upload=103522]: Waiting for error group managing segments... Aug 12 20:51:52.367460826 (99h25m33.098728456s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 20:51:52.367464623 (99h25m33.098732244s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 20:51:52.367467188 (99h25m33.098734818s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 20:51:52.369981799 (99h25m33.10124942s) [upload=103522]: Done uploading segments... Aug 12 20:51:52.369988672 (99h25m33.101256292s) [upload=103522]: Upload finished. err: metaclient: Too Many Requests
Duration: 171.831768ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
424.599037ms
FAIL (2025-08-12 21:01:16 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 12 20:22:01.339537006 (98h55m42.070804636s) [upload=103006]: Starting upload
Aug 12 20:22:01.339603582 (98h55m42.070871202s) [upload=103006]: Done starting upload
Aug 12 20:22:01.339679935 (98h55m42.070947565s) [upload=103006]: Uploading segments...
Aug 12 20:22:01.339853221 (98h55m42.071120851s) [upload=103006]: Got next segment. Inline: false
Aug 12 20:22:01.339892445 (98h55m42.071160075s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 12 20:22:01.339896492 (98h55m42.071164122s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 12 20:22:01.379548451 (98h55m42.110816081s) [seg_pos={0 0}, upload=103006]: Begin upload segment...
Aug 12 20:22:01.379584901 (98h55m42.110852531s) [seg_pos={0 0}, upload=103006]: Done begin upload segment.
Aug 12 20:22:01.379611641 (98h55m42.110879271s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.379628362 (98h55m42.110895992s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380012595 (98h55m42.111280215s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380023856 (98h55m42.111291476s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380277303 (98h55m42.111544923s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380294425 (98h55m42.111562045s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380557831 (98h55m42.111825482s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381005865 (98h55m42.112273495s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381223243 (98h55m42.112490873s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381495867 (98h55m42.112763487s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381860323 (98h55m42.113127953s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382135862 (98h55m42.113403482s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382450184 (98h55m42.113717804s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382711605 (98h55m42.113979225s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382963689 (98h55m42.11423131s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.383222817 (98h55m42.114490438s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384120417 (98h55m42.115388047s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384281039 (98h55m42.115548659s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.503410855 (98h55m42.234678495s) [upload=103006]: Next returned nil segment
Aug 12 20:22:01.503425643 (98h55m42.234693273s) [upload=103006]: Waiting for error group managing segments...
Aug 12 20:22:01.518591716 (98h55m42.249859346s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.522295799 (98h55m42.253563429s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.524942810 (98h55m42.25621044s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529281676 (98h55m42.260549297s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529534613 (98h55m42.260802233s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529663065 (98h55m42.260930685s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.532249323 (98h55m42.263516953s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.537316489 (98h55m42.268584109s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.540779567 (98h55m42.272047197s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.550597055 (98h55m42.281864685s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.552054648 (98h55m42.283322299s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.557838805 (98h55m42.289106454s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.560393523 (98h55m42.291661173s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562560110 (98h55m42.29382776s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562600397 (98h55m42.293868037s) [seg_pos={0 0}, upload=103006]: Segment reached optimal threshold of 14 pieces.
Aug 12 20:22:01.562929186 (98h55m42.294196835s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.562988978 (98h55m42.294256627s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HSvQVuCNaPdj84e4ybE8YWZ77ff2gbhnjd4z9HTf1abGxz2u7): protocol: read tcp 5.161.89.252:22078->79.127.205.235:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563045493 (98h55m42.294313144s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17z3AiUfbxBX2eMDnUnqo1bbgojuQxJro7PsEaPjYzQjHQ3D6K): protocol: read tcp 5.161.89.252:35864->64.49.13.169:4713: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563079208 (98h55m42.294346858s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.563135343 (98h55m42.294402994s) [seg_pos={0 0}, upload=103006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 12 20:22:01.563200666 (98h55m42.294468316s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 12 20:22:01.563205484 (98h55m42.294473135s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 12 20:22:02.476106688 (98h55m43.207374368s) [upload=103006]: Done uploading segments...
Aug 12 20:22:02.476119001 (98h55m43.207386621s) [upload=103006]: Upload finished. err: <nil>
Duration: 424.623052ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..................X............................X............] duration:
227.498276ms
FAIL (2025-08-12 21:00:51 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 20:51:52.367293922 (99h25m33.098561542s) [upload=103522]: Starting upload Aug 12 20:51:52.367341782 (99h25m33.098609412s) [upload=103522]: Done starting upload Aug 12 20:51:52.367390824 (99h25m33.098658444s) [upload=103522]: Uploading segments... Aug 12 20:51:52.367420861 (99h25m33.098688482s) [upload=103522]: Got next segment. Inline: true Aug 12 20:51:52.367424648 (99h25m33.098692278s) [upload=103522]: Waiting for error group managing segments... Aug 12 20:51:52.367460826 (99h25m33.098728456s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 20:51:52.367464623 (99h25m33.098732244s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 20:51:52.367467188 (99h25m33.098734818s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 20:51:52.369981799 (99h25m33.10124942s) [upload=103522]: Done uploading segments... Aug 12 20:51:52.369988672 (99h25m33.101256292s) [upload=103522]: Upload finished. err: metaclient: Too Many Requests
Duration: 227.520588ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
487.679879ms
FAIL (2025-08-12 21:00:44 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 12 20:22:01.339537006 (98h55m42.070804636s) [upload=103006]: Starting upload
Aug 12 20:22:01.339603582 (98h55m42.070871202s) [upload=103006]: Done starting upload
Aug 12 20:22:01.339679935 (98h55m42.070947565s) [upload=103006]: Uploading segments...
Aug 12 20:22:01.339853221 (98h55m42.071120851s) [upload=103006]: Got next segment. Inline: false
Aug 12 20:22:01.339892445 (98h55m42.071160075s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 12 20:22:01.339896492 (98h55m42.071164122s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 12 20:22:01.379548451 (98h55m42.110816081s) [seg_pos={0 0}, upload=103006]: Begin upload segment...
Aug 12 20:22:01.379584901 (98h55m42.110852531s) [seg_pos={0 0}, upload=103006]: Done begin upload segment.
Aug 12 20:22:01.379611641 (98h55m42.110879271s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.379628362 (98h55m42.110895992s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380012595 (98h55m42.111280215s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380023856 (98h55m42.111291476s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380277303 (98h55m42.111544923s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380294425 (98h55m42.111562045s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380557831 (98h55m42.111825482s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381005865 (98h55m42.112273495s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381223243 (98h55m42.112490873s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381495867 (98h55m42.112763487s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381860323 (98h55m42.113127953s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382135862 (98h55m42.113403482s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382450184 (98h55m42.113717804s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382711605 (98h55m42.113979225s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382963689 (98h55m42.11423131s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.383222817 (98h55m42.114490438s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384120417 (98h55m42.115388047s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384281039 (98h55m42.115548659s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.503410855 (98h55m42.234678495s) [upload=103006]: Next returned nil segment
Aug 12 20:22:01.503425643 (98h55m42.234693273s) [upload=103006]: Waiting for error group managing segments...
Aug 12 20:22:01.518591716 (98h55m42.249859346s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.522295799 (98h55m42.253563429s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.524942810 (98h55m42.25621044s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529281676 (98h55m42.260549297s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529534613 (98h55m42.260802233s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529663065 (98h55m42.260930685s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.532249323 (98h55m42.263516953s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.537316489 (98h55m42.268584109s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.540779567 (98h55m42.272047197s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.550597055 (98h55m42.281864685s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.552054648 (98h55m42.283322299s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.557838805 (98h55m42.289106454s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.560393523 (98h55m42.291661173s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562560110 (98h55m42.29382776s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562600397 (98h55m42.293868037s) [seg_pos={0 0}, upload=103006]: Segment reached optimal threshold of 14 pieces.
Aug 12 20:22:01.562929186 (98h55m42.294196835s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.562988978 (98h55m42.294256627s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HSvQVuCNaPdj84e4ybE8YWZ77ff2gbhnjd4z9HTf1abGxz2u7): protocol: read tcp 5.161.89.252:22078->79.127.205.235:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563045493 (98h55m42.294313144s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17z3AiUfbxBX2eMDnUnqo1bbgojuQxJro7PsEaPjYzQjHQ3D6K): protocol: read tcp 5.161.89.252:35864->64.49.13.169:4713: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563079208 (98h55m42.294346858s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.563135343 (98h55m42.294402994s) [seg_pos={0 0}, upload=103006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 12 20:22:01.563200666 (98h55m42.294468316s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 12 20:22:01.563205484 (98h55m42.294473135s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 12 20:22:02.476106688 (98h55m43.207374368s) [upload=103006]: Done uploading segments...
Aug 12 20:22:02.476119001 (98h55m43.207386621s) [upload=103006]: Upload finished. err: <nil>
Duration: 487.708532ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
463.07125ms
FAIL (2025-08-12 21:00:13 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 12 20:22:01.339537006 (98h55m42.070804636s) [upload=103006]: Starting upload
Aug 12 20:22:01.339603582 (98h55m42.070871202s) [upload=103006]: Done starting upload
Aug 12 20:22:01.339679935 (98h55m42.070947565s) [upload=103006]: Uploading segments...
Aug 12 20:22:01.339853221 (98h55m42.071120851s) [upload=103006]: Got next segment. Inline: false
Aug 12 20:22:01.339892445 (98h55m42.071160075s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 12 20:22:01.339896492 (98h55m42.071164122s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 12 20:22:01.379548451 (98h55m42.110816081s) [seg_pos={0 0}, upload=103006]: Begin upload segment...
Aug 12 20:22:01.379584901 (98h55m42.110852531s) [seg_pos={0 0}, upload=103006]: Done begin upload segment.
Aug 12 20:22:01.379611641 (98h55m42.110879271s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.379628362 (98h55m42.110895992s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380012595 (98h55m42.111280215s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380023856 (98h55m42.111291476s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380277303 (98h55m42.111544923s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380294425 (98h55m42.111562045s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380557831 (98h55m42.111825482s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381005865 (98h55m42.112273495s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381223243 (98h55m42.112490873s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381495867 (98h55m42.112763487s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381860323 (98h55m42.113127953s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382135862 (98h55m42.113403482s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382450184 (98h55m42.113717804s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382711605 (98h55m42.113979225s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382963689 (98h55m42.11423131s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.383222817 (98h55m42.114490438s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384120417 (98h55m42.115388047s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384281039 (98h55m42.115548659s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.503410855 (98h55m42.234678495s) [upload=103006]: Next returned nil segment
Aug 12 20:22:01.503425643 (98h55m42.234693273s) [upload=103006]: Waiting for error group managing segments...
Aug 12 20:22:01.518591716 (98h55m42.249859346s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.522295799 (98h55m42.253563429s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.524942810 (98h55m42.25621044s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529281676 (98h55m42.260549297s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529534613 (98h55m42.260802233s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529663065 (98h55m42.260930685s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.532249323 (98h55m42.263516953s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.537316489 (98h55m42.268584109s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.540779567 (98h55m42.272047197s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.550597055 (98h55m42.281864685s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.552054648 (98h55m42.283322299s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.557838805 (98h55m42.289106454s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.560393523 (98h55m42.291661173s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562560110 (98h55m42.29382776s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562600397 (98h55m42.293868037s) [seg_pos={0 0}, upload=103006]: Segment reached optimal threshold of 14 pieces.
Aug 12 20:22:01.562929186 (98h55m42.294196835s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.562988978 (98h55m42.294256627s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HSvQVuCNaPdj84e4ybE8YWZ77ff2gbhnjd4z9HTf1abGxz2u7): protocol: read tcp 5.161.89.252:22078->79.127.205.235:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563045493 (98h55m42.294313144s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17z3AiUfbxBX2eMDnUnqo1bbgojuQxJro7PsEaPjYzQjHQ3D6K): protocol: read tcp 5.161.89.252:35864->64.49.13.169:4713: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563079208 (98h55m42.294346858s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.563135343 (98h55m42.294402994s) [seg_pos={0 0}, upload=103006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 12 20:22:01.563200666 (98h55m42.294468316s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 12 20:22:01.563205484 (98h55m42.294473135s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 12 20:22:02.476106688 (98h55m43.207374368s) [upload=103006]: Done uploading segments...
Aug 12 20:22:02.476119001 (98h55m43.207386621s) [upload=103006]: Upload finished. err: <nil>
Duration: 463.115174ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...................X............................X...........] duration:
156.60634ms
FAIL (2025-08-12 21:00:01 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 20:51:52.367293922 (99h25m33.098561542s) [upload=103522]: Starting upload Aug 12 20:51:52.367341782 (99h25m33.098609412s) [upload=103522]: Done starting upload Aug 12 20:51:52.367390824 (99h25m33.098658444s) [upload=103522]: Uploading segments... Aug 12 20:51:52.367420861 (99h25m33.098688482s) [upload=103522]: Got next segment. Inline: true Aug 12 20:51:52.367424648 (99h25m33.098692278s) [upload=103522]: Waiting for error group managing segments... Aug 12 20:51:52.367460826 (99h25m33.098728456s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 20:51:52.367464623 (99h25m33.098732244s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 20:51:52.367467188 (99h25m33.098734818s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 20:51:52.369981799 (99h25m33.10124942s) [upload=103522]: Done uploading segments... Aug 12 20:51:52.369988672 (99h25m33.101256292s) [upload=103522]: Upload finished. err: metaclient: Too Many Requests
Duration: 156.626136ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
618.323784ms
FAIL (2025-08-12 20:59:35 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 12 20:22:01.339537006 (98h55m42.070804636s) [upload=103006]: Starting upload
Aug 12 20:22:01.339603582 (98h55m42.070871202s) [upload=103006]: Done starting upload
Aug 12 20:22:01.339679935 (98h55m42.070947565s) [upload=103006]: Uploading segments...
Aug 12 20:22:01.339853221 (98h55m42.071120851s) [upload=103006]: Got next segment. Inline: false
Aug 12 20:22:01.339892445 (98h55m42.071160075s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 12 20:22:01.339896492 (98h55m42.071164122s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 12 20:22:01.379548451 (98h55m42.110816081s) [seg_pos={0 0}, upload=103006]: Begin upload segment...
Aug 12 20:22:01.379584901 (98h55m42.110852531s) [seg_pos={0 0}, upload=103006]: Done begin upload segment.
Aug 12 20:22:01.379611641 (98h55m42.110879271s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.379628362 (98h55m42.110895992s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380012595 (98h55m42.111280215s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380023856 (98h55m42.111291476s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380277303 (98h55m42.111544923s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380294425 (98h55m42.111562045s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380557831 (98h55m42.111825482s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381005865 (98h55m42.112273495s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381223243 (98h55m42.112490873s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381495867 (98h55m42.112763487s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381860323 (98h55m42.113127953s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382135862 (98h55m42.113403482s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382450184 (98h55m42.113717804s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382711605 (98h55m42.113979225s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382963689 (98h55m42.11423131s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.383222817 (98h55m42.114490438s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384120417 (98h55m42.115388047s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384281039 (98h55m42.115548659s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.503410855 (98h55m42.234678495s) [upload=103006]: Next returned nil segment
Aug 12 20:22:01.503425643 (98h55m42.234693273s) [upload=103006]: Waiting for error group managing segments...
Aug 12 20:22:01.518591716 (98h55m42.249859346s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.522295799 (98h55m42.253563429s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.524942810 (98h55m42.25621044s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529281676 (98h55m42.260549297s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529534613 (98h55m42.260802233s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529663065 (98h55m42.260930685s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.532249323 (98h55m42.263516953s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.537316489 (98h55m42.268584109s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.540779567 (98h55m42.272047197s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.550597055 (98h55m42.281864685s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.552054648 (98h55m42.283322299s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.557838805 (98h55m42.289106454s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.560393523 (98h55m42.291661173s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562560110 (98h55m42.29382776s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562600397 (98h55m42.293868037s) [seg_pos={0 0}, upload=103006]: Segment reached optimal threshold of 14 pieces.
Aug 12 20:22:01.562929186 (98h55m42.294196835s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.562988978 (98h55m42.294256627s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HSvQVuCNaPdj84e4ybE8YWZ77ff2gbhnjd4z9HTf1abGxz2u7): protocol: read tcp 5.161.89.252:22078->79.127.205.235:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563045493 (98h55m42.294313144s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17z3AiUfbxBX2eMDnUnqo1bbgojuQxJro7PsEaPjYzQjHQ3D6K): protocol: read tcp 5.161.89.252:35864->64.49.13.169:4713: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563079208 (98h55m42.294346858s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.563135343 (98h55m42.294402994s) [seg_pos={0 0}, upload=103006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 12 20:22:01.563200666 (98h55m42.294468316s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 12 20:22:01.563205484 (98h55m42.294473135s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 12 20:22:02.476106688 (98h55m43.207374368s) [upload=103006]: Done uploading segments...
Aug 12 20:22:02.476119001 (98h55m43.207386621s) [upload=103006]: Upload finished. err: <nil>
Duration: 618.343951ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [....................X............................X..........] duration:
336.465887ms
FAIL (2025-08-12 20:59:25 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 20:51:52.367293922 (99h25m33.098561542s) [upload=103522]: Starting upload Aug 12 20:51:52.367341782 (99h25m33.098609412s) [upload=103522]: Done starting upload Aug 12 20:51:52.367390824 (99h25m33.098658444s) [upload=103522]: Uploading segments... Aug 12 20:51:52.367420861 (99h25m33.098688482s) [upload=103522]: Got next segment. Inline: true Aug 12 20:51:52.367424648 (99h25m33.098692278s) [upload=103522]: Waiting for error group managing segments... Aug 12 20:51:52.367460826 (99h25m33.098728456s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 20:51:52.367464623 (99h25m33.098732244s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 20:51:52.367467188 (99h25m33.098734818s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 20:51:52.369981799 (99h25m33.10124942s) [upload=103522]: Done uploading segments... Aug 12 20:51:52.369988672 (99h25m33.101256292s) [upload=103522]: Upload finished. err: metaclient: Too Many Requests
Duration: 336.487758ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
496.500463ms
FAIL (2025-08-12 20:59:16 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 12 20:22:01.339537006 (98h55m42.070804636s) [upload=103006]: Starting upload
Aug 12 20:22:01.339603582 (98h55m42.070871202s) [upload=103006]: Done starting upload
Aug 12 20:22:01.339679935 (98h55m42.070947565s) [upload=103006]: Uploading segments...
Aug 12 20:22:01.339853221 (98h55m42.071120851s) [upload=103006]: Got next segment. Inline: false
Aug 12 20:22:01.339892445 (98h55m42.071160075s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 12 20:22:01.339896492 (98h55m42.071164122s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 12 20:22:01.379548451 (98h55m42.110816081s) [seg_pos={0 0}, upload=103006]: Begin upload segment...
Aug 12 20:22:01.379584901 (98h55m42.110852531s) [seg_pos={0 0}, upload=103006]: Done begin upload segment.
Aug 12 20:22:01.379611641 (98h55m42.110879271s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.379628362 (98h55m42.110895992s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380012595 (98h55m42.111280215s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380023856 (98h55m42.111291476s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380277303 (98h55m42.111544923s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380294425 (98h55m42.111562045s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380557831 (98h55m42.111825482s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381005865 (98h55m42.112273495s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381223243 (98h55m42.112490873s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381495867 (98h55m42.112763487s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381860323 (98h55m42.113127953s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382135862 (98h55m42.113403482s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382450184 (98h55m42.113717804s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382711605 (98h55m42.113979225s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382963689 (98h55m42.11423131s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.383222817 (98h55m42.114490438s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384120417 (98h55m42.115388047s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384281039 (98h55m42.115548659s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.503410855 (98h55m42.234678495s) [upload=103006]: Next returned nil segment
Aug 12 20:22:01.503425643 (98h55m42.234693273s) [upload=103006]: Waiting for error group managing segments...
Aug 12 20:22:01.518591716 (98h55m42.249859346s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.522295799 (98h55m42.253563429s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.524942810 (98h55m42.25621044s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529281676 (98h55m42.260549297s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529534613 (98h55m42.260802233s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529663065 (98h55m42.260930685s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.532249323 (98h55m42.263516953s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.537316489 (98h55m42.268584109s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.540779567 (98h55m42.272047197s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.550597055 (98h55m42.281864685s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.552054648 (98h55m42.283322299s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.557838805 (98h55m42.289106454s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.560393523 (98h55m42.291661173s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562560110 (98h55m42.29382776s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562600397 (98h55m42.293868037s) [seg_pos={0 0}, upload=103006]: Segment reached optimal threshold of 14 pieces.
Aug 12 20:22:01.562929186 (98h55m42.294196835s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.562988978 (98h55m42.294256627s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HSvQVuCNaPdj84e4ybE8YWZ77ff2gbhnjd4z9HTf1abGxz2u7): protocol: read tcp 5.161.89.252:22078->79.127.205.235:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563045493 (98h55m42.294313144s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17z3AiUfbxBX2eMDnUnqo1bbgojuQxJro7PsEaPjYzQjHQ3D6K): protocol: read tcp 5.161.89.252:35864->64.49.13.169:4713: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563079208 (98h55m42.294346858s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.563135343 (98h55m42.294402994s) [seg_pos={0 0}, upload=103006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 12 20:22:01.563200666 (98h55m42.294468316s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 12 20:22:01.563205484 (98h55m42.294473135s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 12 20:22:02.476106688 (98h55m43.207374368s) [upload=103006]: Done uploading segments...
Aug 12 20:22:02.476119001 (98h55m43.207386621s) [upload=103006]: Upload finished. err: <nil>
Duration: 496.531792ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
930.247188ms
FAIL (2025-08-12 20:58:53 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 12 20:22:01.339537006 (98h55m42.070804636s) [upload=103006]: Starting upload
Aug 12 20:22:01.339603582 (98h55m42.070871202s) [upload=103006]: Done starting upload
Aug 12 20:22:01.339679935 (98h55m42.070947565s) [upload=103006]: Uploading segments...
Aug 12 20:22:01.339853221 (98h55m42.071120851s) [upload=103006]: Got next segment. Inline: false
Aug 12 20:22:01.339892445 (98h55m42.071160075s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 12 20:22:01.339896492 (98h55m42.071164122s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 12 20:22:01.379548451 (98h55m42.110816081s) [seg_pos={0 0}, upload=103006]: Begin upload segment...
Aug 12 20:22:01.379584901 (98h55m42.110852531s) [seg_pos={0 0}, upload=103006]: Done begin upload segment.
Aug 12 20:22:01.379611641 (98h55m42.110879271s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.379628362 (98h55m42.110895992s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380012595 (98h55m42.111280215s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380023856 (98h55m42.111291476s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380277303 (98h55m42.111544923s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380294425 (98h55m42.111562045s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380557831 (98h55m42.111825482s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381005865 (98h55m42.112273495s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381223243 (98h55m42.112490873s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381495867 (98h55m42.112763487s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381860323 (98h55m42.113127953s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382135862 (98h55m42.113403482s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382450184 (98h55m42.113717804s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382711605 (98h55m42.113979225s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382963689 (98h55m42.11423131s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.383222817 (98h55m42.114490438s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384120417 (98h55m42.115388047s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384281039 (98h55m42.115548659s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.503410855 (98h55m42.234678495s) [upload=103006]: Next returned nil segment
Aug 12 20:22:01.503425643 (98h55m42.234693273s) [upload=103006]: Waiting for error group managing segments...
Aug 12 20:22:01.518591716 (98h55m42.249859346s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.522295799 (98h55m42.253563429s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.524942810 (98h55m42.25621044s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529281676 (98h55m42.260549297s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529534613 (98h55m42.260802233s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529663065 (98h55m42.260930685s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.532249323 (98h55m42.263516953s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.537316489 (98h55m42.268584109s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.540779567 (98h55m42.272047197s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.550597055 (98h55m42.281864685s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.552054648 (98h55m42.283322299s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.557838805 (98h55m42.289106454s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.560393523 (98h55m42.291661173s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562560110 (98h55m42.29382776s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562600397 (98h55m42.293868037s) [seg_pos={0 0}, upload=103006]: Segment reached optimal threshold of 14 pieces.
Aug 12 20:22:01.562929186 (98h55m42.294196835s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.562988978 (98h55m42.294256627s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HSvQVuCNaPdj84e4ybE8YWZ77ff2gbhnjd4z9HTf1abGxz2u7): protocol: read tcp 5.161.89.252:22078->79.127.205.235:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563045493 (98h55m42.294313144s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17z3AiUfbxBX2eMDnUnqo1bbgojuQxJro7PsEaPjYzQjHQ3D6K): protocol: read tcp 5.161.89.252:35864->64.49.13.169:4713: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563079208 (98h55m42.294346858s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.563135343 (98h55m42.294402994s) [seg_pos={0 0}, upload=103006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 12 20:22:01.563200666 (98h55m42.294468316s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 12 20:22:01.563205484 (98h55m42.294473135s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 12 20:22:02.476106688 (98h55m43.207374368s) [upload=103006]: Done uploading segments...
Aug 12 20:22:02.476119001 (98h55m43.207386621s) [upload=103006]: Upload finished. err: <nil>
Duration: 930.277374ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
529.745534ms
FAIL (2025-08-12 20:58:26 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 12 20:22:01.339537006 (98h55m42.070804636s) [upload=103006]: Starting upload
Aug 12 20:22:01.339603582 (98h55m42.070871202s) [upload=103006]: Done starting upload
Aug 12 20:22:01.339679935 (98h55m42.070947565s) [upload=103006]: Uploading segments...
Aug 12 20:22:01.339853221 (98h55m42.071120851s) [upload=103006]: Got next segment. Inline: false
Aug 12 20:22:01.339892445 (98h55m42.071160075s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 12 20:22:01.339896492 (98h55m42.071164122s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 12 20:22:01.379548451 (98h55m42.110816081s) [seg_pos={0 0}, upload=103006]: Begin upload segment...
Aug 12 20:22:01.379584901 (98h55m42.110852531s) [seg_pos={0 0}, upload=103006]: Done begin upload segment.
Aug 12 20:22:01.379611641 (98h55m42.110879271s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.379628362 (98h55m42.110895992s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380012595 (98h55m42.111280215s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380023856 (98h55m42.111291476s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380277303 (98h55m42.111544923s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380294425 (98h55m42.111562045s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380557831 (98h55m42.111825482s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381005865 (98h55m42.112273495s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381223243 (98h55m42.112490873s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381495867 (98h55m42.112763487s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381860323 (98h55m42.113127953s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382135862 (98h55m42.113403482s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382450184 (98h55m42.113717804s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382711605 (98h55m42.113979225s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382963689 (98h55m42.11423131s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.383222817 (98h55m42.114490438s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384120417 (98h55m42.115388047s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384281039 (98h55m42.115548659s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.503410855 (98h55m42.234678495s) [upload=103006]: Next returned nil segment
Aug 12 20:22:01.503425643 (98h55m42.234693273s) [upload=103006]: Waiting for error group managing segments...
Aug 12 20:22:01.518591716 (98h55m42.249859346s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.522295799 (98h55m42.253563429s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.524942810 (98h55m42.25621044s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529281676 (98h55m42.260549297s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529534613 (98h55m42.260802233s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529663065 (98h55m42.260930685s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.532249323 (98h55m42.263516953s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.537316489 (98h55m42.268584109s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.540779567 (98h55m42.272047197s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.550597055 (98h55m42.281864685s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.552054648 (98h55m42.283322299s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.557838805 (98h55m42.289106454s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.560393523 (98h55m42.291661173s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562560110 (98h55m42.29382776s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562600397 (98h55m42.293868037s) [seg_pos={0 0}, upload=103006]: Segment reached optimal threshold of 14 pieces.
Aug 12 20:22:01.562929186 (98h55m42.294196835s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.562988978 (98h55m42.294256627s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HSvQVuCNaPdj84e4ybE8YWZ77ff2gbhnjd4z9HTf1abGxz2u7): protocol: read tcp 5.161.89.252:22078->79.127.205.235:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563045493 (98h55m42.294313144s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17z3AiUfbxBX2eMDnUnqo1bbgojuQxJro7PsEaPjYzQjHQ3D6K): protocol: read tcp 5.161.89.252:35864->64.49.13.169:4713: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563079208 (98h55m42.294346858s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.563135343 (98h55m42.294402994s) [seg_pos={0 0}, upload=103006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 12 20:22:01.563200666 (98h55m42.294468316s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 12 20:22:01.563205484 (98h55m42.294473135s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 12 20:22:02.476106688 (98h55m43.207374368s) [upload=103006]: Done uploading segments...
Aug 12 20:22:02.476119001 (98h55m43.207386621s) [upload=103006]: Upload finished. err: <nil>
Duration: 529.768776ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.....................X............................X.........] duration:
326.138532ms
FAIL (2025-08-12 20:58:11 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 20:51:52.367293922 (99h25m33.098561542s) [upload=103522]: Starting upload Aug 12 20:51:52.367341782 (99h25m33.098609412s) [upload=103522]: Done starting upload Aug 12 20:51:52.367390824 (99h25m33.098658444s) [upload=103522]: Uploading segments... Aug 12 20:51:52.367420861 (99h25m33.098688482s) [upload=103522]: Got next segment. Inline: true Aug 12 20:51:52.367424648 (99h25m33.098692278s) [upload=103522]: Waiting for error group managing segments... Aug 12 20:51:52.367460826 (99h25m33.098728456s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 20:51:52.367464623 (99h25m33.098732244s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 20:51:52.367467188 (99h25m33.098734818s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 20:51:52.369981799 (99h25m33.10124942s) [upload=103522]: Done uploading segments... Aug 12 20:51:52.369988672 (99h25m33.101256292s) [upload=103522]: Upload finished. err: metaclient: Too Many Requests
Duration: 326.163399ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [......................X............................X........] duration:
218.404368ms
FAIL (2025-08-12 20:57:54 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 20:51:52.367293922 (99h25m33.098561542s) [upload=103522]: Starting upload Aug 12 20:51:52.367341782 (99h25m33.098609412s) [upload=103522]: Done starting upload Aug 12 20:51:52.367390824 (99h25m33.098658444s) [upload=103522]: Uploading segments... Aug 12 20:51:52.367420861 (99h25m33.098688482s) [upload=103522]: Got next segment. Inline: true Aug 12 20:51:52.367424648 (99h25m33.098692278s) [upload=103522]: Waiting for error group managing segments... Aug 12 20:51:52.367460826 (99h25m33.098728456s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 20:51:52.367464623 (99h25m33.098732244s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 20:51:52.367467188 (99h25m33.098734818s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 20:51:52.369981799 (99h25m33.10124942s) [upload=103522]: Done uploading segments... Aug 12 20:51:52.369988672 (99h25m33.101256292s) [upload=103522]: Upload finished. err: metaclient: Too Many Requests
Duration: 218.425999ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
690.138951ms
FAIL (2025-08-12 20:57:30 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 12 20:22:01.339537006 (98h55m42.070804636s) [upload=103006]: Starting upload
Aug 12 20:22:01.339603582 (98h55m42.070871202s) [upload=103006]: Done starting upload
Aug 12 20:22:01.339679935 (98h55m42.070947565s) [upload=103006]: Uploading segments...
Aug 12 20:22:01.339853221 (98h55m42.071120851s) [upload=103006]: Got next segment. Inline: false
Aug 12 20:22:01.339892445 (98h55m42.071160075s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 12 20:22:01.339896492 (98h55m42.071164122s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 12 20:22:01.379548451 (98h55m42.110816081s) [seg_pos={0 0}, upload=103006]: Begin upload segment...
Aug 12 20:22:01.379584901 (98h55m42.110852531s) [seg_pos={0 0}, upload=103006]: Done begin upload segment.
Aug 12 20:22:01.379611641 (98h55m42.110879271s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.379628362 (98h55m42.110895992s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380012595 (98h55m42.111280215s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380023856 (98h55m42.111291476s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380277303 (98h55m42.111544923s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380294425 (98h55m42.111562045s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380557831 (98h55m42.111825482s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381005865 (98h55m42.112273495s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381223243 (98h55m42.112490873s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381495867 (98h55m42.112763487s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381860323 (98h55m42.113127953s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382135862 (98h55m42.113403482s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382450184 (98h55m42.113717804s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382711605 (98h55m42.113979225s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382963689 (98h55m42.11423131s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.383222817 (98h55m42.114490438s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384120417 (98h55m42.115388047s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384281039 (98h55m42.115548659s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.503410855 (98h55m42.234678495s) [upload=103006]: Next returned nil segment
Aug 12 20:22:01.503425643 (98h55m42.234693273s) [upload=103006]: Waiting for error group managing segments...
Aug 12 20:22:01.518591716 (98h55m42.249859346s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.522295799 (98h55m42.253563429s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.524942810 (98h55m42.25621044s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529281676 (98h55m42.260549297s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529534613 (98h55m42.260802233s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529663065 (98h55m42.260930685s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.532249323 (98h55m42.263516953s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.537316489 (98h55m42.268584109s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.540779567 (98h55m42.272047197s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.550597055 (98h55m42.281864685s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.552054648 (98h55m42.283322299s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.557838805 (98h55m42.289106454s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.560393523 (98h55m42.291661173s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562560110 (98h55m42.29382776s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562600397 (98h55m42.293868037s) [seg_pos={0 0}, upload=103006]: Segment reached optimal threshold of 14 pieces.
Aug 12 20:22:01.562929186 (98h55m42.294196835s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.562988978 (98h55m42.294256627s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HSvQVuCNaPdj84e4ybE8YWZ77ff2gbhnjd4z9HTf1abGxz2u7): protocol: read tcp 5.161.89.252:22078->79.127.205.235:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563045493 (98h55m42.294313144s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17z3AiUfbxBX2eMDnUnqo1bbgojuQxJro7PsEaPjYzQjHQ3D6K): protocol: read tcp 5.161.89.252:35864->64.49.13.169:4713: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563079208 (98h55m42.294346858s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.563135343 (98h55m42.294402994s) [seg_pos={0 0}, upload=103006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 12 20:22:01.563200666 (98h55m42.294468316s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 12 20:22:01.563205484 (98h55m42.294473135s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 12 20:22:02.476106688 (98h55m43.207374368s) [upload=103006]: Done uploading segments...
Aug 12 20:22:02.476119001 (98h55m43.207386621s) [upload=103006]: Upload finished. err: <nil>
Duration: 690.159941ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.......................X............................X.......] duration:
155.240239ms
FAIL (2025-08-12 20:57:23 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 20:51:52.367293922 (99h25m33.098561542s) [upload=103522]: Starting upload Aug 12 20:51:52.367341782 (99h25m33.098609412s) [upload=103522]: Done starting upload Aug 12 20:51:52.367390824 (99h25m33.098658444s) [upload=103522]: Uploading segments... Aug 12 20:51:52.367420861 (99h25m33.098688482s) [upload=103522]: Got next segment. Inline: true Aug 12 20:51:52.367424648 (99h25m33.098692278s) [upload=103522]: Waiting for error group managing segments... Aug 12 20:51:52.367460826 (99h25m33.098728456s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 20:51:52.367464623 (99h25m33.098732244s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 20:51:52.367467188 (99h25m33.098734818s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 20:51:52.369981799 (99h25m33.10124942s) [upload=103522]: Done uploading segments... Aug 12 20:51:52.369988672 (99h25m33.101256292s) [upload=103522]: Upload finished. err: metaclient: Too Many Requests
Duration: 155.261941ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [........................X............................X......] duration:
176.514973ms
FAIL (2025-08-12 20:56:43 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 20:51:52.367293922 (99h25m33.098561542s) [upload=103522]: Starting upload Aug 12 20:51:52.367341782 (99h25m33.098609412s) [upload=103522]: Done starting upload Aug 12 20:51:52.367390824 (99h25m33.098658444s) [upload=103522]: Uploading segments... Aug 12 20:51:52.367420861 (99h25m33.098688482s) [upload=103522]: Got next segment. Inline: true Aug 12 20:51:52.367424648 (99h25m33.098692278s) [upload=103522]: Waiting for error group managing segments... Aug 12 20:51:52.367460826 (99h25m33.098728456s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 20:51:52.367464623 (99h25m33.098732244s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 20:51:52.367467188 (99h25m33.098734818s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 20:51:52.369981799 (99h25m33.10124942s) [upload=103522]: Done uploading segments... Aug 12 20:51:52.369988672 (99h25m33.101256292s) [upload=103522]: Upload finished. err: metaclient: Too Many Requests
Duration: 176.530922ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
401.78083ms
FAIL (2025-08-12 20:56:30 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 12 20:22:01.339537006 (98h55m42.070804636s) [upload=103006]: Starting upload
Aug 12 20:22:01.339603582 (98h55m42.070871202s) [upload=103006]: Done starting upload
Aug 12 20:22:01.339679935 (98h55m42.070947565s) [upload=103006]: Uploading segments...
Aug 12 20:22:01.339853221 (98h55m42.071120851s) [upload=103006]: Got next segment. Inline: false
Aug 12 20:22:01.339892445 (98h55m42.071160075s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 12 20:22:01.339896492 (98h55m42.071164122s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 12 20:22:01.379548451 (98h55m42.110816081s) [seg_pos={0 0}, upload=103006]: Begin upload segment...
Aug 12 20:22:01.379584901 (98h55m42.110852531s) [seg_pos={0 0}, upload=103006]: Done begin upload segment.
Aug 12 20:22:01.379611641 (98h55m42.110879271s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.379628362 (98h55m42.110895992s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380012595 (98h55m42.111280215s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380023856 (98h55m42.111291476s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380277303 (98h55m42.111544923s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380294425 (98h55m42.111562045s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380557831 (98h55m42.111825482s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381005865 (98h55m42.112273495s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381223243 (98h55m42.112490873s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381495867 (98h55m42.112763487s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381860323 (98h55m42.113127953s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382135862 (98h55m42.113403482s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382450184 (98h55m42.113717804s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382711605 (98h55m42.113979225s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382963689 (98h55m42.11423131s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.383222817 (98h55m42.114490438s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384120417 (98h55m42.115388047s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384281039 (98h55m42.115548659s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.503410855 (98h55m42.234678495s) [upload=103006]: Next returned nil segment
Aug 12 20:22:01.503425643 (98h55m42.234693273s) [upload=103006]: Waiting for error group managing segments...
Aug 12 20:22:01.518591716 (98h55m42.249859346s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.522295799 (98h55m42.253563429s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.524942810 (98h55m42.25621044s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529281676 (98h55m42.260549297s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529534613 (98h55m42.260802233s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529663065 (98h55m42.260930685s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.532249323 (98h55m42.263516953s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.537316489 (98h55m42.268584109s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.540779567 (98h55m42.272047197s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.550597055 (98h55m42.281864685s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.552054648 (98h55m42.283322299s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.557838805 (98h55m42.289106454s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.560393523 (98h55m42.291661173s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562560110 (98h55m42.29382776s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562600397 (98h55m42.293868037s) [seg_pos={0 0}, upload=103006]: Segment reached optimal threshold of 14 pieces.
Aug 12 20:22:01.562929186 (98h55m42.294196835s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.562988978 (98h55m42.294256627s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HSvQVuCNaPdj84e4ybE8YWZ77ff2gbhnjd4z9HTf1abGxz2u7): protocol: read tcp 5.161.89.252:22078->79.127.205.235:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563045493 (98h55m42.294313144s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17z3AiUfbxBX2eMDnUnqo1bbgojuQxJro7PsEaPjYzQjHQ3D6K): protocol: read tcp 5.161.89.252:35864->64.49.13.169:4713: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563079208 (98h55m42.294346858s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.563135343 (98h55m42.294402994s) [seg_pos={0 0}, upload=103006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 12 20:22:01.563200666 (98h55m42.294468316s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 12 20:22:01.563205484 (98h55m42.294473135s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 12 20:22:02.476106688 (98h55m43.207374368s) [upload=103006]: Done uploading segments...
Aug 12 20:22:02.476119001 (98h55m43.207386621s) [upload=103006]: Upload finished. err: <nil>
Duration: 401.799666ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.........................X............................X.....] duration:
179.4311ms
FAIL (2025-08-12 20:56:19 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 20:51:52.367293922 (99h25m33.098561542s) [upload=103522]: Starting upload Aug 12 20:51:52.367341782 (99h25m33.098609412s) [upload=103522]: Done starting upload Aug 12 20:51:52.367390824 (99h25m33.098658444s) [upload=103522]: Uploading segments... Aug 12 20:51:52.367420861 (99h25m33.098688482s) [upload=103522]: Got next segment. Inline: true Aug 12 20:51:52.367424648 (99h25m33.098692278s) [upload=103522]: Waiting for error group managing segments... Aug 12 20:51:52.367460826 (99h25m33.098728456s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 20:51:52.367464623 (99h25m33.098732244s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 20:51:52.367467188 (99h25m33.098734818s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 20:51:52.369981799 (99h25m33.10124942s) [upload=103522]: Done uploading segments... Aug 12 20:51:52.369988672 (99h25m33.101256292s) [upload=103522]: Upload finished. err: metaclient: Too Many Requests
Duration: 179.455115ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..........................X............................X....] duration:
174.937433ms
FAIL (2025-08-12 20:55:50 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 20:51:52.367293922 (99h25m33.098561542s) [upload=103522]: Starting upload Aug 12 20:51:52.367341782 (99h25m33.098609412s) [upload=103522]: Done starting upload Aug 12 20:51:52.367390824 (99h25m33.098658444s) [upload=103522]: Uploading segments... Aug 12 20:51:52.367420861 (99h25m33.098688482s) [upload=103522]: Got next segment. Inline: true Aug 12 20:51:52.367424648 (99h25m33.098692278s) [upload=103522]: Waiting for error group managing segments... Aug 12 20:51:52.367460826 (99h25m33.098728456s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 20:51:52.367464623 (99h25m33.098732244s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 20:51:52.367467188 (99h25m33.098734818s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 20:51:52.369981799 (99h25m33.10124942s) [upload=103522]: Done uploading segments... Aug 12 20:51:52.369988672 (99h25m33.101256292s) [upload=103522]: Upload finished. err: metaclient: Too Many Requests
Duration: 174.960157ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
630.75739ms
FAIL (2025-08-12 20:55:46 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 12 20:22:01.339537006 (98h55m42.070804636s) [upload=103006]: Starting upload
Aug 12 20:22:01.339603582 (98h55m42.070871202s) [upload=103006]: Done starting upload
Aug 12 20:22:01.339679935 (98h55m42.070947565s) [upload=103006]: Uploading segments...
Aug 12 20:22:01.339853221 (98h55m42.071120851s) [upload=103006]: Got next segment. Inline: false
Aug 12 20:22:01.339892445 (98h55m42.071160075s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 12 20:22:01.339896492 (98h55m42.071164122s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 12 20:22:01.379548451 (98h55m42.110816081s) [seg_pos={0 0}, upload=103006]: Begin upload segment...
Aug 12 20:22:01.379584901 (98h55m42.110852531s) [seg_pos={0 0}, upload=103006]: Done begin upload segment.
Aug 12 20:22:01.379611641 (98h55m42.110879271s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.379628362 (98h55m42.110895992s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380012595 (98h55m42.111280215s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380023856 (98h55m42.111291476s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380277303 (98h55m42.111544923s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380294425 (98h55m42.111562045s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380557831 (98h55m42.111825482s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381005865 (98h55m42.112273495s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381223243 (98h55m42.112490873s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381495867 (98h55m42.112763487s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381860323 (98h55m42.113127953s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382135862 (98h55m42.113403482s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382450184 (98h55m42.113717804s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382711605 (98h55m42.113979225s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382963689 (98h55m42.11423131s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.383222817 (98h55m42.114490438s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384120417 (98h55m42.115388047s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384281039 (98h55m42.115548659s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.503410855 (98h55m42.234678495s) [upload=103006]: Next returned nil segment
Aug 12 20:22:01.503425643 (98h55m42.234693273s) [upload=103006]: Waiting for error group managing segments...
Aug 12 20:22:01.518591716 (98h55m42.249859346s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.522295799 (98h55m42.253563429s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.524942810 (98h55m42.25621044s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529281676 (98h55m42.260549297s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529534613 (98h55m42.260802233s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529663065 (98h55m42.260930685s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.532249323 (98h55m42.263516953s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.537316489 (98h55m42.268584109s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.540779567 (98h55m42.272047197s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.550597055 (98h55m42.281864685s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.552054648 (98h55m42.283322299s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.557838805 (98h55m42.289106454s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.560393523 (98h55m42.291661173s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562560110 (98h55m42.29382776s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562600397 (98h55m42.293868037s) [seg_pos={0 0}, upload=103006]: Segment reached optimal threshold of 14 pieces.
Aug 12 20:22:01.562929186 (98h55m42.294196835s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.562988978 (98h55m42.294256627s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HSvQVuCNaPdj84e4ybE8YWZ77ff2gbhnjd4z9HTf1abGxz2u7): protocol: read tcp 5.161.89.252:22078->79.127.205.235:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563045493 (98h55m42.294313144s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17z3AiUfbxBX2eMDnUnqo1bbgojuQxJro7PsEaPjYzQjHQ3D6K): protocol: read tcp 5.161.89.252:35864->64.49.13.169:4713: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563079208 (98h55m42.294346858s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.563135343 (98h55m42.294402994s) [seg_pos={0 0}, upload=103006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 12 20:22:01.563200666 (98h55m42.294468316s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 12 20:22:01.563205484 (98h55m42.294473135s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 12 20:22:02.476106688 (98h55m43.207374368s) [upload=103006]: Done uploading segments...
Aug 12 20:22:02.476119001 (98h55m43.207386621s) [upload=103006]: Upload finished. err: <nil>
Duration: 630.77362ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
430.068588ms
FAIL (2025-08-12 20:55:17 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 12 20:22:01.339537006 (98h55m42.070804636s) [upload=103006]: Starting upload
Aug 12 20:22:01.339603582 (98h55m42.070871202s) [upload=103006]: Done starting upload
Aug 12 20:22:01.339679935 (98h55m42.070947565s) [upload=103006]: Uploading segments...
Aug 12 20:22:01.339853221 (98h55m42.071120851s) [upload=103006]: Got next segment. Inline: false
Aug 12 20:22:01.339892445 (98h55m42.071160075s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 12 20:22:01.339896492 (98h55m42.071164122s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 12 20:22:01.379548451 (98h55m42.110816081s) [seg_pos={0 0}, upload=103006]: Begin upload segment...
Aug 12 20:22:01.379584901 (98h55m42.110852531s) [seg_pos={0 0}, upload=103006]: Done begin upload segment.
Aug 12 20:22:01.379611641 (98h55m42.110879271s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.379628362 (98h55m42.110895992s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380012595 (98h55m42.111280215s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380023856 (98h55m42.111291476s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380277303 (98h55m42.111544923s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380294425 (98h55m42.111562045s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380557831 (98h55m42.111825482s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381005865 (98h55m42.112273495s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381223243 (98h55m42.112490873s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381495867 (98h55m42.112763487s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381860323 (98h55m42.113127953s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382135862 (98h55m42.113403482s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382450184 (98h55m42.113717804s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382711605 (98h55m42.113979225s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382963689 (98h55m42.11423131s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.383222817 (98h55m42.114490438s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384120417 (98h55m42.115388047s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384281039 (98h55m42.115548659s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.503410855 (98h55m42.234678495s) [upload=103006]: Next returned nil segment
Aug 12 20:22:01.503425643 (98h55m42.234693273s) [upload=103006]: Waiting for error group managing segments...
Aug 12 20:22:01.518591716 (98h55m42.249859346s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.522295799 (98h55m42.253563429s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.524942810 (98h55m42.25621044s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529281676 (98h55m42.260549297s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529534613 (98h55m42.260802233s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529663065 (98h55m42.260930685s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.532249323 (98h55m42.263516953s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.537316489 (98h55m42.268584109s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.540779567 (98h55m42.272047197s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.550597055 (98h55m42.281864685s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.552054648 (98h55m42.283322299s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.557838805 (98h55m42.289106454s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.560393523 (98h55m42.291661173s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562560110 (98h55m42.29382776s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562600397 (98h55m42.293868037s) [seg_pos={0 0}, upload=103006]: Segment reached optimal threshold of 14 pieces.
Aug 12 20:22:01.562929186 (98h55m42.294196835s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.562988978 (98h55m42.294256627s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HSvQVuCNaPdj84e4ybE8YWZ77ff2gbhnjd4z9HTf1abGxz2u7): protocol: read tcp 5.161.89.252:22078->79.127.205.235:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563045493 (98h55m42.294313144s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17z3AiUfbxBX2eMDnUnqo1bbgojuQxJro7PsEaPjYzQjHQ3D6K): protocol: read tcp 5.161.89.252:35864->64.49.13.169:4713: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563079208 (98h55m42.294346858s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.563135343 (98h55m42.294402994s) [seg_pos={0 0}, upload=103006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 12 20:22:01.563200666 (98h55m42.294468316s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 12 20:22:01.563205484 (98h55m42.294473135s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 12 20:22:02.476106688 (98h55m43.207374368s) [upload=103006]: Done uploading segments...
Aug 12 20:22:02.476119001 (98h55m43.207386621s) [upload=103006]: Upload finished. err: <nil>
Duration: 430.0954ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...........................X............................X...] duration:
170.641167ms
FAIL (2025-08-12 20:54:46 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 20:51:52.367293922 (99h25m33.098561542s) [upload=103522]: Starting upload Aug 12 20:51:52.367341782 (99h25m33.098609412s) [upload=103522]: Done starting upload Aug 12 20:51:52.367390824 (99h25m33.098658444s) [upload=103522]: Uploading segments... Aug 12 20:51:52.367420861 (99h25m33.098688482s) [upload=103522]: Got next segment. Inline: true Aug 12 20:51:52.367424648 (99h25m33.098692278s) [upload=103522]: Waiting for error group managing segments... Aug 12 20:51:52.367460826 (99h25m33.098728456s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 20:51:52.367464623 (99h25m33.098732244s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 20:51:52.367467188 (99h25m33.098734818s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 20:51:52.369981799 (99h25m33.10124942s) [upload=103522]: Done uploading segments... Aug 12 20:51:52.369988672 (99h25m33.101256292s) [upload=103522]: Upload finished. err: metaclient: Too Many Requests
Duration: 170.660835ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
905.171271ms
FAIL (2025-08-12 20:54:35 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 12 20:22:01.339537006 (98h55m42.070804636s) [upload=103006]: Starting upload
Aug 12 20:22:01.339603582 (98h55m42.070871202s) [upload=103006]: Done starting upload
Aug 12 20:22:01.339679935 (98h55m42.070947565s) [upload=103006]: Uploading segments...
Aug 12 20:22:01.339853221 (98h55m42.071120851s) [upload=103006]: Got next segment. Inline: false
Aug 12 20:22:01.339892445 (98h55m42.071160075s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 12 20:22:01.339896492 (98h55m42.071164122s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 12 20:22:01.379548451 (98h55m42.110816081s) [seg_pos={0 0}, upload=103006]: Begin upload segment...
Aug 12 20:22:01.379584901 (98h55m42.110852531s) [seg_pos={0 0}, upload=103006]: Done begin upload segment.
Aug 12 20:22:01.379611641 (98h55m42.110879271s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.379628362 (98h55m42.110895992s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380012595 (98h55m42.111280215s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380023856 (98h55m42.111291476s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380277303 (98h55m42.111544923s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380294425 (98h55m42.111562045s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380557831 (98h55m42.111825482s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381005865 (98h55m42.112273495s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381223243 (98h55m42.112490873s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381495867 (98h55m42.112763487s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381860323 (98h55m42.113127953s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382135862 (98h55m42.113403482s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382450184 (98h55m42.113717804s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382711605 (98h55m42.113979225s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382963689 (98h55m42.11423131s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.383222817 (98h55m42.114490438s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384120417 (98h55m42.115388047s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384281039 (98h55m42.115548659s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.503410855 (98h55m42.234678495s) [upload=103006]: Next returned nil segment
Aug 12 20:22:01.503425643 (98h55m42.234693273s) [upload=103006]: Waiting for error group managing segments...
Aug 12 20:22:01.518591716 (98h55m42.249859346s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.522295799 (98h55m42.253563429s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.524942810 (98h55m42.25621044s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529281676 (98h55m42.260549297s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529534613 (98h55m42.260802233s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529663065 (98h55m42.260930685s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.532249323 (98h55m42.263516953s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.537316489 (98h55m42.268584109s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.540779567 (98h55m42.272047197s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.550597055 (98h55m42.281864685s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.552054648 (98h55m42.283322299s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.557838805 (98h55m42.289106454s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.560393523 (98h55m42.291661173s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562560110 (98h55m42.29382776s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562600397 (98h55m42.293868037s) [seg_pos={0 0}, upload=103006]: Segment reached optimal threshold of 14 pieces.
Aug 12 20:22:01.562929186 (98h55m42.294196835s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.562988978 (98h55m42.294256627s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HSvQVuCNaPdj84e4ybE8YWZ77ff2gbhnjd4z9HTf1abGxz2u7): protocol: read tcp 5.161.89.252:22078->79.127.205.235:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563045493 (98h55m42.294313144s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17z3AiUfbxBX2eMDnUnqo1bbgojuQxJro7PsEaPjYzQjHQ3D6K): protocol: read tcp 5.161.89.252:35864->64.49.13.169:4713: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563079208 (98h55m42.294346858s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.563135343 (98h55m42.294402994s) [seg_pos={0 0}, upload=103006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 12 20:22:01.563200666 (98h55m42.294468316s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 12 20:22:01.563205484 (98h55m42.294473135s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 12 20:22:02.476106688 (98h55m43.207374368s) [upload=103006]: Done uploading segments...
Aug 12 20:22:02.476119001 (98h55m43.207386621s) [upload=103006]: Upload finished. err: <nil>
Duration: 905.196437ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
390.481598ms
FAIL (2025-08-12 20:54:14 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 12 20:22:01.339537006 (98h55m42.070804636s) [upload=103006]: Starting upload
Aug 12 20:22:01.339603582 (98h55m42.070871202s) [upload=103006]: Done starting upload
Aug 12 20:22:01.339679935 (98h55m42.070947565s) [upload=103006]: Uploading segments...
Aug 12 20:22:01.339853221 (98h55m42.071120851s) [upload=103006]: Got next segment. Inline: false
Aug 12 20:22:01.339892445 (98h55m42.071160075s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 12 20:22:01.339896492 (98h55m42.071164122s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 12 20:22:01.379548451 (98h55m42.110816081s) [seg_pos={0 0}, upload=103006]: Begin upload segment...
Aug 12 20:22:01.379584901 (98h55m42.110852531s) [seg_pos={0 0}, upload=103006]: Done begin upload segment.
Aug 12 20:22:01.379611641 (98h55m42.110879271s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.379628362 (98h55m42.110895992s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380012595 (98h55m42.111280215s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380023856 (98h55m42.111291476s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380277303 (98h55m42.111544923s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380294425 (98h55m42.111562045s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380557831 (98h55m42.111825482s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381005865 (98h55m42.112273495s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381223243 (98h55m42.112490873s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381495867 (98h55m42.112763487s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381860323 (98h55m42.113127953s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382135862 (98h55m42.113403482s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382450184 (98h55m42.113717804s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382711605 (98h55m42.113979225s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382963689 (98h55m42.11423131s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.383222817 (98h55m42.114490438s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384120417 (98h55m42.115388047s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384281039 (98h55m42.115548659s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.503410855 (98h55m42.234678495s) [upload=103006]: Next returned nil segment
Aug 12 20:22:01.503425643 (98h55m42.234693273s) [upload=103006]: Waiting for error group managing segments...
Aug 12 20:22:01.518591716 (98h55m42.249859346s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.522295799 (98h55m42.253563429s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.524942810 (98h55m42.25621044s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529281676 (98h55m42.260549297s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529534613 (98h55m42.260802233s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529663065 (98h55m42.260930685s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.532249323 (98h55m42.263516953s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.537316489 (98h55m42.268584109s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.540779567 (98h55m42.272047197s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.550597055 (98h55m42.281864685s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.552054648 (98h55m42.283322299s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.557838805 (98h55m42.289106454s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.560393523 (98h55m42.291661173s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562560110 (98h55m42.29382776s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562600397 (98h55m42.293868037s) [seg_pos={0 0}, upload=103006]: Segment reached optimal threshold of 14 pieces.
Aug 12 20:22:01.562929186 (98h55m42.294196835s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.562988978 (98h55m42.294256627s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HSvQVuCNaPdj84e4ybE8YWZ77ff2gbhnjd4z9HTf1abGxz2u7): protocol: read tcp 5.161.89.252:22078->79.127.205.235:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563045493 (98h55m42.294313144s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17z3AiUfbxBX2eMDnUnqo1bbgojuQxJro7PsEaPjYzQjHQ3D6K): protocol: read tcp 5.161.89.252:35864->64.49.13.169:4713: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563079208 (98h55m42.294346858s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.563135343 (98h55m42.294402994s) [seg_pos={0 0}, upload=103006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 12 20:22:01.563200666 (98h55m42.294468316s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 12 20:22:01.563205484 (98h55m42.294473135s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 12 20:22:02.476106688 (98h55m43.207374368s) [upload=103006]: Done uploading segments...
Aug 12 20:22:02.476119001 (98h55m43.207386621s) [upload=103006]: Upload finished. err: <nil>
Duration: 390.503831ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [............................X............................X..] duration:
153.527454ms
FAIL (2025-08-12 20:53:50 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 20:51:52.367293922 (99h25m33.098561542s) [upload=103522]: Starting upload Aug 12 20:51:52.367341782 (99h25m33.098609412s) [upload=103522]: Done starting upload Aug 12 20:51:52.367390824 (99h25m33.098658444s) [upload=103522]: Uploading segments... Aug 12 20:51:52.367420861 (99h25m33.098688482s) [upload=103522]: Got next segment. Inline: true Aug 12 20:51:52.367424648 (99h25m33.098692278s) [upload=103522]: Waiting for error group managing segments... Aug 12 20:51:52.367460826 (99h25m33.098728456s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 20:51:52.367464623 (99h25m33.098732244s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 20:51:52.367467188 (99h25m33.098734818s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 20:51:52.369981799 (99h25m33.10124942s) [upload=103522]: Done uploading segments... Aug 12 20:51:52.369988672 (99h25m33.101256292s) [upload=103522]: Upload finished. err: metaclient: Too Many Requests
Duration: 153.572431ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
683.014643ms
FAIL (2025-08-12 20:53:14 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 12 20:22:01.339537006 (98h55m42.070804636s) [upload=103006]: Starting upload
Aug 12 20:22:01.339603582 (98h55m42.070871202s) [upload=103006]: Done starting upload
Aug 12 20:22:01.339679935 (98h55m42.070947565s) [upload=103006]: Uploading segments...
Aug 12 20:22:01.339853221 (98h55m42.071120851s) [upload=103006]: Got next segment. Inline: false
Aug 12 20:22:01.339892445 (98h55m42.071160075s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 12 20:22:01.339896492 (98h55m42.071164122s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 12 20:22:01.379548451 (98h55m42.110816081s) [seg_pos={0 0}, upload=103006]: Begin upload segment...
Aug 12 20:22:01.379584901 (98h55m42.110852531s) [seg_pos={0 0}, upload=103006]: Done begin upload segment.
Aug 12 20:22:01.379611641 (98h55m42.110879271s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.379628362 (98h55m42.110895992s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380012595 (98h55m42.111280215s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380023856 (98h55m42.111291476s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380277303 (98h55m42.111544923s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380294425 (98h55m42.111562045s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380557831 (98h55m42.111825482s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381005865 (98h55m42.112273495s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381223243 (98h55m42.112490873s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381495867 (98h55m42.112763487s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381860323 (98h55m42.113127953s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382135862 (98h55m42.113403482s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382450184 (98h55m42.113717804s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382711605 (98h55m42.113979225s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382963689 (98h55m42.11423131s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.383222817 (98h55m42.114490438s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384120417 (98h55m42.115388047s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384281039 (98h55m42.115548659s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.503410855 (98h55m42.234678495s) [upload=103006]: Next returned nil segment
Aug 12 20:22:01.503425643 (98h55m42.234693273s) [upload=103006]: Waiting for error group managing segments...
Aug 12 20:22:01.518591716 (98h55m42.249859346s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.522295799 (98h55m42.253563429s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.524942810 (98h55m42.25621044s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529281676 (98h55m42.260549297s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529534613 (98h55m42.260802233s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529663065 (98h55m42.260930685s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.532249323 (98h55m42.263516953s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.537316489 (98h55m42.268584109s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.540779567 (98h55m42.272047197s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.550597055 (98h55m42.281864685s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.552054648 (98h55m42.283322299s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.557838805 (98h55m42.289106454s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.560393523 (98h55m42.291661173s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562560110 (98h55m42.29382776s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562600397 (98h55m42.293868037s) [seg_pos={0 0}, upload=103006]: Segment reached optimal threshold of 14 pieces.
Aug 12 20:22:01.562929186 (98h55m42.294196835s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.562988978 (98h55m42.294256627s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HSvQVuCNaPdj84e4ybE8YWZ77ff2gbhnjd4z9HTf1abGxz2u7): protocol: read tcp 5.161.89.252:22078->79.127.205.235:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563045493 (98h55m42.294313144s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17z3AiUfbxBX2eMDnUnqo1bbgojuQxJro7PsEaPjYzQjHQ3D6K): protocol: read tcp 5.161.89.252:35864->64.49.13.169:4713: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563079208 (98h55m42.294346858s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.563135343 (98h55m42.294402994s) [seg_pos={0 0}, upload=103006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 12 20:22:01.563200666 (98h55m42.294468316s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 12 20:22:01.563205484 (98h55m42.294473135s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 12 20:22:02.476106688 (98h55m43.207374368s) [upload=103006]: Done uploading segments...
Aug 12 20:22:02.476119001 (98h55m43.207386621s) [upload=103006]: Upload finished. err: <nil>
Duration: 683.037225ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.............................X............................X.] duration:
230.363248ms
FAIL (2025-08-12 20:52:36 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 20:51:52.367293922 (99h25m33.098561542s) [upload=103522]: Starting upload Aug 12 20:51:52.367341782 (99h25m33.098609412s) [upload=103522]: Done starting upload Aug 12 20:51:52.367390824 (99h25m33.098658444s) [upload=103522]: Uploading segments... Aug 12 20:51:52.367420861 (99h25m33.098688482s) [upload=103522]: Got next segment. Inline: true Aug 12 20:51:52.367424648 (99h25m33.098692278s) [upload=103522]: Waiting for error group managing segments... Aug 12 20:51:52.367460826 (99h25m33.098728456s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 20:51:52.367464623 (99h25m33.098732244s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 20:51:52.367467188 (99h25m33.098734818s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 20:51:52.369981799 (99h25m33.10124942s) [upload=103522]: Done uploading segments... Aug 12 20:51:52.369988672 (99h25m33.101256292s) [upload=103522]: Upload finished. err: metaclient: Too Many Requests
Duration: 230.452987ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
428.493675ms
FAIL (2025-08-12 20:52:19 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 12 20:22:01.339537006 (98h55m42.070804636s) [upload=103006]: Starting upload
Aug 12 20:22:01.339603582 (98h55m42.070871202s) [upload=103006]: Done starting upload
Aug 12 20:22:01.339679935 (98h55m42.070947565s) [upload=103006]: Uploading segments...
Aug 12 20:22:01.339853221 (98h55m42.071120851s) [upload=103006]: Got next segment. Inline: false
Aug 12 20:22:01.339892445 (98h55m42.071160075s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 12 20:22:01.339896492 (98h55m42.071164122s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 12 20:22:01.379548451 (98h55m42.110816081s) [seg_pos={0 0}, upload=103006]: Begin upload segment...
Aug 12 20:22:01.379584901 (98h55m42.110852531s) [seg_pos={0 0}, upload=103006]: Done begin upload segment.
Aug 12 20:22:01.379611641 (98h55m42.110879271s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.379628362 (98h55m42.110895992s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380012595 (98h55m42.111280215s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380023856 (98h55m42.111291476s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380277303 (98h55m42.111544923s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380294425 (98h55m42.111562045s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380557831 (98h55m42.111825482s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381005865 (98h55m42.112273495s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381223243 (98h55m42.112490873s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381495867 (98h55m42.112763487s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381860323 (98h55m42.113127953s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382135862 (98h55m42.113403482s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382450184 (98h55m42.113717804s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382711605 (98h55m42.113979225s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382963689 (98h55m42.11423131s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.383222817 (98h55m42.114490438s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384120417 (98h55m42.115388047s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384281039 (98h55m42.115548659s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.503410855 (98h55m42.234678495s) [upload=103006]: Next returned nil segment
Aug 12 20:22:01.503425643 (98h55m42.234693273s) [upload=103006]: Waiting for error group managing segments...
Aug 12 20:22:01.518591716 (98h55m42.249859346s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.522295799 (98h55m42.253563429s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.524942810 (98h55m42.25621044s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529281676 (98h55m42.260549297s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529534613 (98h55m42.260802233s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529663065 (98h55m42.260930685s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.532249323 (98h55m42.263516953s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.537316489 (98h55m42.268584109s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.540779567 (98h55m42.272047197s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.550597055 (98h55m42.281864685s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.552054648 (98h55m42.283322299s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.557838805 (98h55m42.289106454s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.560393523 (98h55m42.291661173s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562560110 (98h55m42.29382776s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562600397 (98h55m42.293868037s) [seg_pos={0 0}, upload=103006]: Segment reached optimal threshold of 14 pieces.
Aug 12 20:22:01.562929186 (98h55m42.294196835s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.562988978 (98h55m42.294256627s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HSvQVuCNaPdj84e4ybE8YWZ77ff2gbhnjd4z9HTf1abGxz2u7): protocol: read tcp 5.161.89.252:22078->79.127.205.235:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563045493 (98h55m42.294313144s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17z3AiUfbxBX2eMDnUnqo1bbgojuQxJro7PsEaPjYzQjHQ3D6K): protocol: read tcp 5.161.89.252:35864->64.49.13.169:4713: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563079208 (98h55m42.294346858s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.563135343 (98h55m42.294402994s) [seg_pos={0 0}, upload=103006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 12 20:22:01.563200666 (98h55m42.294468316s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 12 20:22:01.563205484 (98h55m42.294473135s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 12 20:22:02.476106688 (98h55m43.207374368s) [upload=103006]: Done uploading segments...
Aug 12 20:22:02.476119001 (98h55m43.207386621s) [upload=103006]: Upload finished. err: <nil>
Duration: 428.524422ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..............................X............................X] duration:
16.897212ms
FAIL (2025-08-12 20:51:52 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 20:51:52.367293922 (99h25m33.098561542s) [upload=103522]: Starting upload Aug 12 20:51:52.367341782 (99h25m33.098609412s) [upload=103522]: Done starting upload Aug 12 20:51:52.367390824 (99h25m33.098658444s) [upload=103522]: Uploading segments... Aug 12 20:51:52.367420861 (99h25m33.098688482s) [upload=103522]: Got next segment. Inline: true Aug 12 20:51:52.367424648 (99h25m33.098692278s) [upload=103522]: Waiting for error group managing segments... Aug 12 20:51:52.367460826 (99h25m33.098728456s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 20:51:52.367464623 (99h25m33.098732244s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 20:51:52.367467188 (99h25m33.098734818s) [upload=103522]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 20:51:52.369981799 (99h25m33.10124942s) [upload=103522]: Done uploading segments... Aug 12 20:51:52.369988672 (99h25m33.101256292s) [upload=103522]: Upload finished. err: metaclient: Too Many Requests
Duration: 16.913882ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
644.134069ms
FAIL (2025-08-12 20:51:26 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 12 20:22:01.339537006 (98h55m42.070804636s) [upload=103006]: Starting upload
Aug 12 20:22:01.339603582 (98h55m42.070871202s) [upload=103006]: Done starting upload
Aug 12 20:22:01.339679935 (98h55m42.070947565s) [upload=103006]: Uploading segments...
Aug 12 20:22:01.339853221 (98h55m42.071120851s) [upload=103006]: Got next segment. Inline: false
Aug 12 20:22:01.339892445 (98h55m42.071160075s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 12 20:22:01.339896492 (98h55m42.071164122s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 12 20:22:01.379548451 (98h55m42.110816081s) [seg_pos={0 0}, upload=103006]: Begin upload segment...
Aug 12 20:22:01.379584901 (98h55m42.110852531s) [seg_pos={0 0}, upload=103006]: Done begin upload segment.
Aug 12 20:22:01.379611641 (98h55m42.110879271s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.379628362 (98h55m42.110895992s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380012595 (98h55m42.111280215s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380023856 (98h55m42.111291476s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380277303 (98h55m42.111544923s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380294425 (98h55m42.111562045s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380557831 (98h55m42.111825482s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381005865 (98h55m42.112273495s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381223243 (98h55m42.112490873s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381495867 (98h55m42.112763487s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381860323 (98h55m42.113127953s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382135862 (98h55m42.113403482s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382450184 (98h55m42.113717804s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382711605 (98h55m42.113979225s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382963689 (98h55m42.11423131s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.383222817 (98h55m42.114490438s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384120417 (98h55m42.115388047s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384281039 (98h55m42.115548659s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.503410855 (98h55m42.234678495s) [upload=103006]: Next returned nil segment
Aug 12 20:22:01.503425643 (98h55m42.234693273s) [upload=103006]: Waiting for error group managing segments...
Aug 12 20:22:01.518591716 (98h55m42.249859346s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.522295799 (98h55m42.253563429s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.524942810 (98h55m42.25621044s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529281676 (98h55m42.260549297s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529534613 (98h55m42.260802233s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529663065 (98h55m42.260930685s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.532249323 (98h55m42.263516953s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.537316489 (98h55m42.268584109s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.540779567 (98h55m42.272047197s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.550597055 (98h55m42.281864685s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.552054648 (98h55m42.283322299s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.557838805 (98h55m42.289106454s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.560393523 (98h55m42.291661173s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562560110 (98h55m42.29382776s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562600397 (98h55m42.293868037s) [seg_pos={0 0}, upload=103006]: Segment reached optimal threshold of 14 pieces.
Aug 12 20:22:01.562929186 (98h55m42.294196835s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.562988978 (98h55m42.294256627s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HSvQVuCNaPdj84e4ybE8YWZ77ff2gbhnjd4z9HTf1abGxz2u7): protocol: read tcp 5.161.89.252:22078->79.127.205.235:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563045493 (98h55m42.294313144s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17z3AiUfbxBX2eMDnUnqo1bbgojuQxJro7PsEaPjYzQjHQ3D6K): protocol: read tcp 5.161.89.252:35864->64.49.13.169:4713: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563079208 (98h55m42.294346858s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.563135343 (98h55m42.294402994s) [seg_pos={0 0}, upload=103006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 12 20:22:01.563200666 (98h55m42.294468316s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 12 20:22:01.563205484 (98h55m42.294473135s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 12 20:22:02.476106688 (98h55m43.207374368s) [upload=103006]: Done uploading segments...
Aug 12 20:22:02.476119001 (98h55m43.207386621s) [upload=103006]: Upload finished. err: <nil>
Duration: 644.163554ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
409.67905ms
FAIL (2025-08-12 20:50:49 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 12 20:22:01.339537006 (98h55m42.070804636s) [upload=103006]: Starting upload
Aug 12 20:22:01.339603582 (98h55m42.070871202s) [upload=103006]: Done starting upload
Aug 12 20:22:01.339679935 (98h55m42.070947565s) [upload=103006]: Uploading segments...
Aug 12 20:22:01.339853221 (98h55m42.071120851s) [upload=103006]: Got next segment. Inline: false
Aug 12 20:22:01.339892445 (98h55m42.071160075s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 12 20:22:01.339896492 (98h55m42.071164122s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 12 20:22:01.379548451 (98h55m42.110816081s) [seg_pos={0 0}, upload=103006]: Begin upload segment...
Aug 12 20:22:01.379584901 (98h55m42.110852531s) [seg_pos={0 0}, upload=103006]: Done begin upload segment.
Aug 12 20:22:01.379611641 (98h55m42.110879271s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.379628362 (98h55m42.110895992s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380012595 (98h55m42.111280215s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380023856 (98h55m42.111291476s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380277303 (98h55m42.111544923s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380294425 (98h55m42.111562045s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380557831 (98h55m42.111825482s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381005865 (98h55m42.112273495s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381223243 (98h55m42.112490873s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381495867 (98h55m42.112763487s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381860323 (98h55m42.113127953s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382135862 (98h55m42.113403482s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382450184 (98h55m42.113717804s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382711605 (98h55m42.113979225s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382963689 (98h55m42.11423131s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.383222817 (98h55m42.114490438s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384120417 (98h55m42.115388047s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384281039 (98h55m42.115548659s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.503410855 (98h55m42.234678495s) [upload=103006]: Next returned nil segment
Aug 12 20:22:01.503425643 (98h55m42.234693273s) [upload=103006]: Waiting for error group managing segments...
Aug 12 20:22:01.518591716 (98h55m42.249859346s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.522295799 (98h55m42.253563429s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.524942810 (98h55m42.25621044s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529281676 (98h55m42.260549297s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529534613 (98h55m42.260802233s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529663065 (98h55m42.260930685s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.532249323 (98h55m42.263516953s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.537316489 (98h55m42.268584109s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.540779567 (98h55m42.272047197s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.550597055 (98h55m42.281864685s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.552054648 (98h55m42.283322299s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.557838805 (98h55m42.289106454s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.560393523 (98h55m42.291661173s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562560110 (98h55m42.29382776s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562600397 (98h55m42.293868037s) [seg_pos={0 0}, upload=103006]: Segment reached optimal threshold of 14 pieces.
Aug 12 20:22:01.562929186 (98h55m42.294196835s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.562988978 (98h55m42.294256627s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HSvQVuCNaPdj84e4ybE8YWZ77ff2gbhnjd4z9HTf1abGxz2u7): protocol: read tcp 5.161.89.252:22078->79.127.205.235:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563045493 (98h55m42.294313144s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17z3AiUfbxBX2eMDnUnqo1bbgojuQxJro7PsEaPjYzQjHQ3D6K): protocol: read tcp 5.161.89.252:35864->64.49.13.169:4713: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563079208 (98h55m42.294346858s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.563135343 (98h55m42.294402994s) [seg_pos={0 0}, upload=103006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 12 20:22:01.563200666 (98h55m42.294468316s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 12 20:22:01.563205484 (98h55m42.294473135s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 12 20:22:02.476106688 (98h55m43.207374368s) [upload=103006]: Done uploading segments...
Aug 12 20:22:02.476119001 (98h55m43.207386621s) [upload=103006]: Upload finished. err: <nil>
Duration: 409.69541ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
164.638945ms
FAIL (2025-08-12 20:50:48 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 20:29:20.443735527 (99h3m1.175003147s) [upload=103132]: Starting upload Aug 12 20:29:20.443814516 (99h3m1.175082136s) [upload=103132]: Done starting upload Aug 12 20:29:20.443953527 (99h3m1.175221157s) [upload=103132]: Uploading segments... Aug 12 20:29:20.444007669 (99h3m1.175275309s) [upload=103132]: Got next segment. Inline: true Aug 12 20:29:20.444012588 (99h3m1.175280208s) [upload=103132]: Waiting for error group managing segments... Aug 12 20:29:20.444062733 (99h3m1.175330352s) [upload=103132]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 20:29:20.444067902 (99h3m1.175335522s) [upload=103132]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 20:29:20.444087398 (99h3m1.175355019s) [upload=103132]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 20:29:20.450757743 (99h3m1.182025363s) [upload=103132]: Done uploading segments... Aug 12 20:29:20.450767311 (99h3m1.182034941s) [upload=103132]: Upload finished. err: metaclient: Too Many Requests
Duration: 164.654785ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
167.810786ms
FAIL (2025-08-12 20:50:20 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 20:29:20.443735527 (99h3m1.175003147s) [upload=103132]: Starting upload Aug 12 20:29:20.443814516 (99h3m1.175082136s) [upload=103132]: Done starting upload Aug 12 20:29:20.443953527 (99h3m1.175221157s) [upload=103132]: Uploading segments... Aug 12 20:29:20.444007669 (99h3m1.175275309s) [upload=103132]: Got next segment. Inline: true Aug 12 20:29:20.444012588 (99h3m1.175280208s) [upload=103132]: Waiting for error group managing segments... Aug 12 20:29:20.444062733 (99h3m1.175330352s) [upload=103132]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 20:29:20.444067902 (99h3m1.175335522s) [upload=103132]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 20:29:20.444087398 (99h3m1.175355019s) [upload=103132]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 20:29:20.450757743 (99h3m1.182025363s) [upload=103132]: Done uploading segments... Aug 12 20:29:20.450767311 (99h3m1.182034941s) [upload=103132]: Upload finished. err: metaclient: Too Many Requests
Duration: 167.83953ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
833.804783ms
FAIL (2025-08-12 20:50:16 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 12 20:22:01.339537006 (98h55m42.070804636s) [upload=103006]: Starting upload
Aug 12 20:22:01.339603582 (98h55m42.070871202s) [upload=103006]: Done starting upload
Aug 12 20:22:01.339679935 (98h55m42.070947565s) [upload=103006]: Uploading segments...
Aug 12 20:22:01.339853221 (98h55m42.071120851s) [upload=103006]: Got next segment. Inline: false
Aug 12 20:22:01.339892445 (98h55m42.071160075s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 12 20:22:01.339896492 (98h55m42.071164122s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 12 20:22:01.379548451 (98h55m42.110816081s) [seg_pos={0 0}, upload=103006]: Begin upload segment...
Aug 12 20:22:01.379584901 (98h55m42.110852531s) [seg_pos={0 0}, upload=103006]: Done begin upload segment.
Aug 12 20:22:01.379611641 (98h55m42.110879271s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.379628362 (98h55m42.110895992s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380012595 (98h55m42.111280215s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380023856 (98h55m42.111291476s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380277303 (98h55m42.111544923s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380294425 (98h55m42.111562045s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380557831 (98h55m42.111825482s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381005865 (98h55m42.112273495s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381223243 (98h55m42.112490873s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381495867 (98h55m42.112763487s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381860323 (98h55m42.113127953s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382135862 (98h55m42.113403482s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382450184 (98h55m42.113717804s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382711605 (98h55m42.113979225s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382963689 (98h55m42.11423131s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.383222817 (98h55m42.114490438s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384120417 (98h55m42.115388047s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384281039 (98h55m42.115548659s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.503410855 (98h55m42.234678495s) [upload=103006]: Next returned nil segment
Aug 12 20:22:01.503425643 (98h55m42.234693273s) [upload=103006]: Waiting for error group managing segments...
Aug 12 20:22:01.518591716 (98h55m42.249859346s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.522295799 (98h55m42.253563429s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.524942810 (98h55m42.25621044s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529281676 (98h55m42.260549297s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529534613 (98h55m42.260802233s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529663065 (98h55m42.260930685s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.532249323 (98h55m42.263516953s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.537316489 (98h55m42.268584109s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.540779567 (98h55m42.272047197s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.550597055 (98h55m42.281864685s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.552054648 (98h55m42.283322299s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.557838805 (98h55m42.289106454s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.560393523 (98h55m42.291661173s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562560110 (98h55m42.29382776s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562600397 (98h55m42.293868037s) [seg_pos={0 0}, upload=103006]: Segment reached optimal threshold of 14 pieces.
Aug 12 20:22:01.562929186 (98h55m42.294196835s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.562988978 (98h55m42.294256627s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HSvQVuCNaPdj84e4ybE8YWZ77ff2gbhnjd4z9HTf1abGxz2u7): protocol: read tcp 5.161.89.252:22078->79.127.205.235:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563045493 (98h55m42.294313144s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17z3AiUfbxBX2eMDnUnqo1bbgojuQxJro7PsEaPjYzQjHQ3D6K): protocol: read tcp 5.161.89.252:35864->64.49.13.169:4713: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563079208 (98h55m42.294346858s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.563135343 (98h55m42.294402994s) [seg_pos={0 0}, upload=103006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 12 20:22:01.563200666 (98h55m42.294468316s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 12 20:22:01.563205484 (98h55m42.294473135s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 12 20:22:02.476106688 (98h55m43.207374368s) [upload=103006]: Done uploading segments...
Aug 12 20:22:02.476119001 (98h55m43.207386621s) [upload=103006]: Upload finished. err: <nil>
Duration: 833.830982ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
509.431426ms
FAIL (2025-08-12 20:49:22 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 12 20:22:01.339537006 (98h55m42.070804636s) [upload=103006]: Starting upload
Aug 12 20:22:01.339603582 (98h55m42.070871202s) [upload=103006]: Done starting upload
Aug 12 20:22:01.339679935 (98h55m42.070947565s) [upload=103006]: Uploading segments...
Aug 12 20:22:01.339853221 (98h55m42.071120851s) [upload=103006]: Got next segment. Inline: false
Aug 12 20:22:01.339892445 (98h55m42.071160075s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 12 20:22:01.339896492 (98h55m42.071164122s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 12 20:22:01.379548451 (98h55m42.110816081s) [seg_pos={0 0}, upload=103006]: Begin upload segment...
Aug 12 20:22:01.379584901 (98h55m42.110852531s) [seg_pos={0 0}, upload=103006]: Done begin upload segment.
Aug 12 20:22:01.379611641 (98h55m42.110879271s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.379628362 (98h55m42.110895992s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380012595 (98h55m42.111280215s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380023856 (98h55m42.111291476s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380277303 (98h55m42.111544923s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380294425 (98h55m42.111562045s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380557831 (98h55m42.111825482s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381005865 (98h55m42.112273495s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381223243 (98h55m42.112490873s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381495867 (98h55m42.112763487s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381860323 (98h55m42.113127953s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382135862 (98h55m42.113403482s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382450184 (98h55m42.113717804s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382711605 (98h55m42.113979225s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382963689 (98h55m42.11423131s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.383222817 (98h55m42.114490438s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384120417 (98h55m42.115388047s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384281039 (98h55m42.115548659s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.503410855 (98h55m42.234678495s) [upload=103006]: Next returned nil segment
Aug 12 20:22:01.503425643 (98h55m42.234693273s) [upload=103006]: Waiting for error group managing segments...
Aug 12 20:22:01.518591716 (98h55m42.249859346s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.522295799 (98h55m42.253563429s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.524942810 (98h55m42.25621044s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529281676 (98h55m42.260549297s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529534613 (98h55m42.260802233s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529663065 (98h55m42.260930685s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.532249323 (98h55m42.263516953s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.537316489 (98h55m42.268584109s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.540779567 (98h55m42.272047197s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.550597055 (98h55m42.281864685s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.552054648 (98h55m42.283322299s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.557838805 (98h55m42.289106454s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.560393523 (98h55m42.291661173s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562560110 (98h55m42.29382776s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562600397 (98h55m42.293868037s) [seg_pos={0 0}, upload=103006]: Segment reached optimal threshold of 14 pieces.
Aug 12 20:22:01.562929186 (98h55m42.294196835s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.562988978 (98h55m42.294256627s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HSvQVuCNaPdj84e4ybE8YWZ77ff2gbhnjd4z9HTf1abGxz2u7): protocol: read tcp 5.161.89.252:22078->79.127.205.235:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563045493 (98h55m42.294313144s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17z3AiUfbxBX2eMDnUnqo1bbgojuQxJro7PsEaPjYzQjHQ3D6K): protocol: read tcp 5.161.89.252:35864->64.49.13.169:4713: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563079208 (98h55m42.294346858s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.563135343 (98h55m42.294402994s) [seg_pos={0 0}, upload=103006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 12 20:22:01.563200666 (98h55m42.294468316s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 12 20:22:01.563205484 (98h55m42.294473135s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 12 20:22:02.476106688 (98h55m43.207374368s) [upload=103006]: Done uploading segments...
Aug 12 20:22:02.476119001 (98h55m43.207386621s) [upload=103006]: Upload finished. err: <nil>
Duration: 509.461461ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
173.916538ms
FAIL (2025-08-12 20:49:18 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 20:29:20.443735527 (99h3m1.175003147s) [upload=103132]: Starting upload Aug 12 20:29:20.443814516 (99h3m1.175082136s) [upload=103132]: Done starting upload Aug 12 20:29:20.443953527 (99h3m1.175221157s) [upload=103132]: Uploading segments... Aug 12 20:29:20.444007669 (99h3m1.175275309s) [upload=103132]: Got next segment. Inline: true Aug 12 20:29:20.444012588 (99h3m1.175280208s) [upload=103132]: Waiting for error group managing segments... Aug 12 20:29:20.444062733 (99h3m1.175330352s) [upload=103132]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 20:29:20.444067902 (99h3m1.175335522s) [upload=103132]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 20:29:20.444087398 (99h3m1.175355019s) [upload=103132]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 20:29:20.450757743 (99h3m1.182025363s) [upload=103132]: Done uploading segments... Aug 12 20:29:20.450767311 (99h3m1.182034941s) [upload=103132]: Upload finished. err: metaclient: Too Many Requests
Duration: 173.939743ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
269.089082ms
FAIL (2025-08-12 20:49:00 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 20:29:20.443735527 (99h3m1.175003147s) [upload=103132]: Starting upload Aug 12 20:29:20.443814516 (99h3m1.175082136s) [upload=103132]: Done starting upload Aug 12 20:29:20.443953527 (99h3m1.175221157s) [upload=103132]: Uploading segments... Aug 12 20:29:20.444007669 (99h3m1.175275309s) [upload=103132]: Got next segment. Inline: true Aug 12 20:29:20.444012588 (99h3m1.175280208s) [upload=103132]: Waiting for error group managing segments... Aug 12 20:29:20.444062733 (99h3m1.175330352s) [upload=103132]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 20:29:20.444067902 (99h3m1.175335522s) [upload=103132]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 20:29:20.444087398 (99h3m1.175355019s) [upload=103132]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 20:29:20.450757743 (99h3m1.182025363s) [upload=103132]: Done uploading segments... Aug 12 20:29:20.450767311 (99h3m1.182034941s) [upload=103132]: Upload finished. err: metaclient: Too Many Requests
Duration: 269.107627ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
590.495453ms
FAIL (2025-08-12 20:48:28 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 12 20:22:01.339537006 (98h55m42.070804636s) [upload=103006]: Starting upload
Aug 12 20:22:01.339603582 (98h55m42.070871202s) [upload=103006]: Done starting upload
Aug 12 20:22:01.339679935 (98h55m42.070947565s) [upload=103006]: Uploading segments...
Aug 12 20:22:01.339853221 (98h55m42.071120851s) [upload=103006]: Got next segment. Inline: false
Aug 12 20:22:01.339892445 (98h55m42.071160075s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 12 20:22:01.339896492 (98h55m42.071164122s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 12 20:22:01.379548451 (98h55m42.110816081s) [seg_pos={0 0}, upload=103006]: Begin upload segment...
Aug 12 20:22:01.379584901 (98h55m42.110852531s) [seg_pos={0 0}, upload=103006]: Done begin upload segment.
Aug 12 20:22:01.379611641 (98h55m42.110879271s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.379628362 (98h55m42.110895992s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380012595 (98h55m42.111280215s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380023856 (98h55m42.111291476s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380277303 (98h55m42.111544923s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380294425 (98h55m42.111562045s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.380557831 (98h55m42.111825482s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381005865 (98h55m42.112273495s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381223243 (98h55m42.112490873s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381495867 (98h55m42.112763487s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.381860323 (98h55m42.113127953s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382135862 (98h55m42.113403482s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382450184 (98h55m42.113717804s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382711605 (98h55m42.113979225s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.382963689 (98h55m42.11423131s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.383222817 (98h55m42.114490438s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384120417 (98h55m42.115388047s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.384281039 (98h55m42.115548659s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Uploading piece...
Aug 12 20:22:01.503410855 (98h55m42.234678495s) [upload=103006]: Next returned nil segment
Aug 12 20:22:01.503425643 (98h55m42.234693273s) [upload=103006]: Waiting for error group managing segments...
Aug 12 20:22:01.518591716 (98h55m42.249859346s) [noise=true , address=104.36.82.39:4754 , piece_id=7ZBCC3Q72O4KIWFZVO2O4U6FS7UL5P67FU3EY2UMW7TFCDUYYE4A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.522295799 (98h55m42.253563429s) [noise=true , address=104.36.82.14:4232 , piece_id=UO3P7FBYFJPBFWZFRKK6CWWMCDEB7YAPC5FPREKSQOEQ2JU6I37Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.524942810 (98h55m42.25621044s) [noise=true , address=104.36.82.8:4993 , piece_id=5JGUV4QMSBG5YAPTQTUOMPTGUCUGQXUKFZGFS32MPDPU3D3G7Y4Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529281676 (98h55m42.260549297s) [noise=true , address=104.36.82.35:4582 , piece_id=DVWEOHMCH7ADDY2RXPKPOE55LOQAE636ZLJ2BRIUFTQX22BRNUKA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529534613 (98h55m42.260802233s) [noise=true , address=104.36.82.40:4972 , piece_id=OJGZIQQ7FBYSKYJTAVGFIULWVZCBUF2HPJVC7TUMEI3HEQJTUNKQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.529663065 (98h55m42.260930685s) [noise=true , address=104.36.82.23:4445 , piece_id=VKVNYY73RMER75AMFM6DSEDLKCZAKEXJQSTMPQOOCKFAXN23OJQA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.532249323 (98h55m42.263516953s) [noise=true , address=104.36.82.62:4420 , piece_id=JUYUQFID5GUDXGCIHMCZUWDDC3WIVR5E72XEGFZUAPFAKBEAJPSQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.537316489 (98h55m42.268584109s) [noise=true , address=104.36.82.36:4618 , piece_id=3BXDRNPSA77ZCRWCFLYYRXAGJTL2O6HZ42GRZPQXM6RA26NJOF6A, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.540779567 (98h55m42.272047197s) [noise=true , address=104.36.82.13:4210 , piece_id=DIQBERJ3UOM2H2FTNK6B7JA3ATUTYR4DZ3A26CMK6TVS5CIVIMXQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.550597055 (98h55m42.281864685s) [noise=true , address=77.90.31.21:4899 , piece_id=XB3VNMD6V6LJJHATXLHW6MGXQX667NYKUTS6FJ7QRP5MVPCFKTDQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.552054648 (98h55m42.283322299s) [noise=true , address=104.36.82.12:4168 , piece_id=4U4ZQQOMMF53XG7WW3CXN6A5YVN2Q4EQXUY6RKI2P45FBOOVSRSA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.557838805 (98h55m42.289106454s) [noise=true , address=104.36.82.9:4035 , piece_id=JI3Y6MFMMFSBL7YDA45H7RBV2FUM7SLFP4LIZ6O2BBW3KB7QFQYA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.560393523 (98h55m42.291661173s) [noise=true , address=104.36.82.62:4436 , piece_id=WSYIG725WIYG6QJ6FO3UWSG6QP6KA2IBDNM6FI6IF2QEOA5WMU3Q, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562560110 (98h55m42.29382776s) [noise=true , address=104.36.82.61:4394 , piece_id=NZMAPLSHTI4TO3D36YMDSPUWNGSJ7GAK4AM5NXTZOMC4YU7P5RWQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: <nil>
Aug 12 20:22:01.562600397 (98h55m42.293868037s) [seg_pos={0 0}, upload=103006]: Segment reached optimal threshold of 14 pieces.
Aug 12 20:22:01.562929186 (98h55m42.294196835s) [noise=true , address=104.36.82.59:4306 , piece_id=7VZX2QOV3S5U3JTBTXGHENIXO3XJKM7CXHQEGECBQTUROERNA6CQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.562988978 (98h55m42.294256627s) [noise=true , address=79.127.205.235:4658 , piece_id=LYEUZ4WBS32YU2L43D7JTSD3NPSZKGPOZMYWFVAWAUR4GYW2XVGA, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HSvQVuCNaPdj84e4ybE8YWZ77ff2gbhnjd4z9HTf1abGxz2u7): protocol: read tcp 5.161.89.252:22078->79.127.205.235:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563045493 (98h55m42.294313144s) [noise=true , address=64.49.13.169:4713 , piece_id=NPVCOAXQ3U2ZDNCXZMU2C356Q2WBI54L6LJHKEM7M276G34YXPIQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17z3AiUfbxBX2eMDnUnqo1bbgojuQxJro7PsEaPjYzQjHQ3D6K): protocol: read tcp 5.161.89.252:35864->64.49.13.169:4713: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 12 20:22:01.563079208 (98h55m42.294346858s) [noise=true , address=104.36.82.21:4506 , piece_id=EWJA4LY7K3QJWZXVITMXDYVGXTPJAIRC2CVLWPXIH46OYABIMONQ, seg_pos={0 0}, upload=103006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T): protocol: expected piece hash; context canceled; EOF
Aug 12 20:22:01.563135343 (98h55m42.294402994s) [seg_pos={0 0}, upload=103006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 12 20:22:01.563200666 (98h55m42.294468316s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 12 20:22:01.563205484 (98h55m42.294473135s) [upload=103006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 12 20:22:02.476106688 (98h55m43.207374368s) [upload=103006]: Done uploading segments...
Aug 12 20:22:02.476119001 (98h55m43.207386621s) [upload=103006]: Upload finished. err: <nil>
Duration: 590.516843ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|