Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
417.9306ms trace:
https://jaeger.dev.storj.io/trace/5ee917a62999ce3b
FAIL (2025-12-07 14:07:35 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec 7 14:05:12.793746279 (46h36m0.491314615s) [upload=40891]: Starting upload
Dec 7 14:05:12.793820377 (46h36m0.491388713s) [upload=40891]: Done starting upload
Dec 7 14:05:12.794040619 (46h36m0.491608935s) [upload=40891]: Uploading segments...
Dec 7 14:05:12.794207350 (46h36m0.491775685s) [upload=40891]: Got next segment. Inline: false
Dec 7 14:05:12.794270879 (46h36m0.491839214s) [upload=40891]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 7 14:05:12.794277000 (46h36m0.491845346s) [upload=40891]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 7 14:05:12.815883299 (46h36m0.513451636s) [seg_pos={0 0}, upload=40891]: Begin upload segment...
Dec 7 14:05:12.815956126 (46h36m0.513524472s) [seg_pos={0 0}, upload=40891]: Done begin upload segment.
Dec 7 14:05:12.815973168 (46h36m0.513541474s) [noise=true , address=145.79.227.29:4941 , piece_id=ROH6OOR3PU6F6YGZMGBDVL7IZI2YFW6WOMALTBPE2DKDBH35D43Q, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.815992825 (46h36m0.513561161s) [noise=true , address=77.90.31.17:4772 , piece_id=5C5RYNJHJOR74O6MNAGKIAHGBZLGT7ZYAPHG4CJI655GS2UFFGCQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.816593907 (46h36m0.514162243s) [noise=true , address=145.79.227.42:4123 , piece_id=ISX3YLJJQ6L4HMTGY2J47EZM74UZJA5M65JN5TOFY6IAA2DHPABQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.817033549 (46h36m0.514601884s) [noise=true , address=145.79.227.32:4834 , piece_id=PD7R4WLJLLZBCFND6BUPEJ3LIJBKNETFMLB2C2X73HH3Q4APZ6KQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.817608943 (46h36m0.515177279s) [noise=true , address=145.79.227.30:4763 , piece_id=TMPQE3SUM2SWP2RO3YGACVUIAVHWV2BUUX522XCSJBACMEQJT2IQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.818051330 (46h36m0.515619666s) [noise=true , address=145.79.227.40:4051 , piece_id=NS2P7FBECXVP2LCCF6UQ4BDDL7IXN4BWLRTIT6MAUOF2C5FVFI3A, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.818512791 (46h36m0.516081127s) [noise=true , address=145.79.227.41:4086 , piece_id=35MLDKAKB7U26I33RKISI33JYRZK36NVREY4H43LQ2GJ7IEEW4EA, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.819185398 (46h36m0.516753734s) [noise=true , address=145.79.227.23:4546 , piece_id=FTUSB2QRIKNNSIN67M5EZYRSM6MUYMC746TLBLZXTYEXLVE7MYTQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.821078003 (46h36m0.518646339s) [noise=true , address=145.79.227.43:4159 , piece_id=R7UD47XJ7XGLARQAFNL2ISONIR6VZLFJ3PWWQ5V4ZKYB7KMD6MCA, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.822854892 (46h36m0.520423238s) [noise=true , address=45.91.116.11:4117 , piece_id=WELLJ6VZNKBHBNFTK7SZZ72J2EKFBVA6SCB27HX7JIMQUYXPY5BA, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.824567812 (46h36m0.522136148s) [noise=true , address=145.79.227.31:4797 , piece_id=6NHGY624TXJ2NL4KGLCKR5TETBSH6R4KF23O3RG3LYR7OOKJ55ZA, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.826399914 (46h36m0.52396823s) [noise=true , address=145.79.227.43:4158 , piece_id=C7JLSPLTCF7SJVKHDYAOTLARPNSDHNU3ARV2FJQMEUR4QZOWOH3Q, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.827587042 (46h36m0.525155388s) [noise=true , address=145.79.227.38:4977 , piece_id=3EAL7Y6XNXCRCNLW2SX7EADFYVVZIWQ4AJ5BQMIQV3KJKCC4C4IQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.828083339 (46h36m0.525651645s) [noise=true , address=145.79.227.27:4689 , piece_id=YDNSB4BUQDP7CNIAU73X6ILCKRWUYQWQEIUA2IYFPLPDGNNJ56YQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.829168395 (46h36m0.526736702s) [noise=true , address=77.90.31.16:4734 , piece_id=EU5C6XZ3NAU7AXJ6T2X3YNE5TLN44UD5OTKHWDW7AMMH2OWUB7QQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.830278419 (46h36m0.527846735s) [noise=true , address=77.90.31.12:4592 , piece_id=WHMBDMM2OKNWDRBIUYQG4N4S4BIHPSA3C2TNAEJ7BGRMDK5G726A, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.876303152 (46h36m0.573871468s) [noise=true , address=77.90.31.16:4734 , piece_id=EU5C6XZ3NAU7AXJ6T2X3YNE5TLN44UD5OTKHWDW7AMMH2OWUB7QQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: storage node overloaded, request limit: 2000
Dec 7 14:05:12.877232387 (46h36m0.574800703s) [noise=true , address=145.79.227.30:4763 , piece_id=TMPQE3SUM2SWP2RO3YGACVUIAVHWV2BUUX522XCSJBACMEQJT2IQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: storage node overloaded, request limit: 2000
Dec 7 14:05:12.916831066 (46h36m0.614399382s) [upload=40891]: Next returned nil segment
Dec 7 14:05:12.916839262 (46h36m0.614407567s) [upload=40891]: Waiting for error group managing segments...
Dec 7 14:05:12.929551988 (46h36m0.627120294s) [noise=true , address=145.79.227.42:4123 , piece_id=ISX3YLJJQ6L4HMTGY2J47EZM74UZJA5M65JN5TOFY6IAA2DHPABQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.929866526 (46h36m0.627434872s) [noise=true , address=145.79.227.27:4689 , piece_id=YDNSB4BUQDP7CNIAU73X6ILCKRWUYQWQEIUA2IYFPLPDGNNJ56YQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.930150587 (46h36m0.627718893s) [noise=true , address=145.79.227.43:4159 , piece_id=R7UD47XJ7XGLARQAFNL2ISONIR6VZLFJ3PWWQ5V4ZKYB7KMD6MCA, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.930672422 (46h36m0.628240757s) [noise=true , address=145.79.227.29:4941 , piece_id=ROH6OOR3PU6F6YGZMGBDVL7IZI2YFW6WOMALTBPE2DKDBH35D43Q, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.930722245 (46h36m0.628290561s) [noise=true , address=145.79.227.41:4086 , piece_id=35MLDKAKB7U26I33RKISI33JYRZK36NVREY4H43LQ2GJ7IEEW4EA, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.931139735 (46h36m0.62870804s) [noise=true , address=145.79.227.32:4834 , piece_id=PD7R4WLJLLZBCFND6BUPEJ3LIJBKNETFMLB2C2X73HH3Q4APZ6KQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.931585557 (46h36m0.629153894s) [noise=true , address=145.79.227.38:4977 , piece_id=3EAL7Y6XNXCRCNLW2SX7EADFYVVZIWQ4AJ5BQMIQV3KJKCC4C4IQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.931807582 (46h36m0.629375888s) [noise=true , address=145.79.227.23:4546 , piece_id=FTUSB2QRIKNNSIN67M5EZYRSM6MUYMC746TLBLZXTYEXLVE7MYTQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.932398866 (46h36m0.629967213s) [noise=true , address=145.79.227.40:4051 , piece_id=NS2P7FBECXVP2LCCF6UQ4BDDL7IXN4BWLRTIT6MAUOF2C5FVFI3A, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.932913437 (46h36m0.630481743s) [noise=true , address=145.79.227.43:4158 , piece_id=C7JLSPLTCF7SJVKHDYAOTLARPNSDHNU3ARV2FJQMEUR4QZOWOH3Q, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.936343114 (46h36m0.63391143s) [noise=true , address=145.79.227.31:4797 , piece_id=6NHGY624TXJ2NL4KGLCKR5TETBSH6R4KF23O3RG3LYR7OOKJ55ZA, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.938164697 (46h36m0.635733033s) [noise=true , address=45.91.116.11:4117 , piece_id=WELLJ6VZNKBHBNFTK7SZZ72J2EKFBVA6SCB27HX7JIMQUYXPY5BA, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:42.780287323 (46h36m30.477855639s) [noise=true , address=77.90.31.17:4772 , piece_id=5C5RYNJHJOR74O6MNAGKIAHGBZLGT7ZYAPHG4CJI655GS2UFFGCQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: expected piece hash; context deadline exceeded; EOF
Dec 7 14:05:42.780406927 (46h36m30.477975233s) [noise=true , address=77.90.31.12:4592 , piece_id=WHMBDMM2OKNWDRBIUYQG4N4S4BIHPSA3C2TNAEJ7BGRMDK5G726A, seg_pos={0 0}, upload=40891]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF
Dec 7 14:05:42.780455096 (46h36m30.478023402s) [seg_pos={0 0}, upload=40891]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec 7 14:05:42.813667787 (46h36m30.511236093s) [upload=40891]: Done uploading segments...
Dec 7 14:05:42.813692484 (46h36m30.5112608s) [upload=40891]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 417.949796ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
342.266172ms trace:
https://jaeger.dev.storj.io/trace/5ee917a62999ce3b
FAIL (2025-12-07 14:06:26 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec 7 14:05:12.793746279 (46h36m0.491314615s) [upload=40891]: Starting upload
Dec 7 14:05:12.793820377 (46h36m0.491388713s) [upload=40891]: Done starting upload
Dec 7 14:05:12.794040619 (46h36m0.491608935s) [upload=40891]: Uploading segments...
Dec 7 14:05:12.794207350 (46h36m0.491775685s) [upload=40891]: Got next segment. Inline: false
Dec 7 14:05:12.794270879 (46h36m0.491839214s) [upload=40891]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 7 14:05:12.794277000 (46h36m0.491845346s) [upload=40891]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 7 14:05:12.815883299 (46h36m0.513451636s) [seg_pos={0 0}, upload=40891]: Begin upload segment...
Dec 7 14:05:12.815956126 (46h36m0.513524472s) [seg_pos={0 0}, upload=40891]: Done begin upload segment.
Dec 7 14:05:12.815973168 (46h36m0.513541474s) [noise=true , address=145.79.227.29:4941 , piece_id=ROH6OOR3PU6F6YGZMGBDVL7IZI2YFW6WOMALTBPE2DKDBH35D43Q, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.815992825 (46h36m0.513561161s) [noise=true , address=77.90.31.17:4772 , piece_id=5C5RYNJHJOR74O6MNAGKIAHGBZLGT7ZYAPHG4CJI655GS2UFFGCQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.816593907 (46h36m0.514162243s) [noise=true , address=145.79.227.42:4123 , piece_id=ISX3YLJJQ6L4HMTGY2J47EZM74UZJA5M65JN5TOFY6IAA2DHPABQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.817033549 (46h36m0.514601884s) [noise=true , address=145.79.227.32:4834 , piece_id=PD7R4WLJLLZBCFND6BUPEJ3LIJBKNETFMLB2C2X73HH3Q4APZ6KQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.817608943 (46h36m0.515177279s) [noise=true , address=145.79.227.30:4763 , piece_id=TMPQE3SUM2SWP2RO3YGACVUIAVHWV2BUUX522XCSJBACMEQJT2IQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.818051330 (46h36m0.515619666s) [noise=true , address=145.79.227.40:4051 , piece_id=NS2P7FBECXVP2LCCF6UQ4BDDL7IXN4BWLRTIT6MAUOF2C5FVFI3A, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.818512791 (46h36m0.516081127s) [noise=true , address=145.79.227.41:4086 , piece_id=35MLDKAKB7U26I33RKISI33JYRZK36NVREY4H43LQ2GJ7IEEW4EA, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.819185398 (46h36m0.516753734s) [noise=true , address=145.79.227.23:4546 , piece_id=FTUSB2QRIKNNSIN67M5EZYRSM6MUYMC746TLBLZXTYEXLVE7MYTQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.821078003 (46h36m0.518646339s) [noise=true , address=145.79.227.43:4159 , piece_id=R7UD47XJ7XGLARQAFNL2ISONIR6VZLFJ3PWWQ5V4ZKYB7KMD6MCA, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.822854892 (46h36m0.520423238s) [noise=true , address=45.91.116.11:4117 , piece_id=WELLJ6VZNKBHBNFTK7SZZ72J2EKFBVA6SCB27HX7JIMQUYXPY5BA, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.824567812 (46h36m0.522136148s) [noise=true , address=145.79.227.31:4797 , piece_id=6NHGY624TXJ2NL4KGLCKR5TETBSH6R4KF23O3RG3LYR7OOKJ55ZA, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.826399914 (46h36m0.52396823s) [noise=true , address=145.79.227.43:4158 , piece_id=C7JLSPLTCF7SJVKHDYAOTLARPNSDHNU3ARV2FJQMEUR4QZOWOH3Q, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.827587042 (46h36m0.525155388s) [noise=true , address=145.79.227.38:4977 , piece_id=3EAL7Y6XNXCRCNLW2SX7EADFYVVZIWQ4AJ5BQMIQV3KJKCC4C4IQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.828083339 (46h36m0.525651645s) [noise=true , address=145.79.227.27:4689 , piece_id=YDNSB4BUQDP7CNIAU73X6ILCKRWUYQWQEIUA2IYFPLPDGNNJ56YQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.829168395 (46h36m0.526736702s) [noise=true , address=77.90.31.16:4734 , piece_id=EU5C6XZ3NAU7AXJ6T2X3YNE5TLN44UD5OTKHWDW7AMMH2OWUB7QQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.830278419 (46h36m0.527846735s) [noise=true , address=77.90.31.12:4592 , piece_id=WHMBDMM2OKNWDRBIUYQG4N4S4BIHPSA3C2TNAEJ7BGRMDK5G726A, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.876303152 (46h36m0.573871468s) [noise=true , address=77.90.31.16:4734 , piece_id=EU5C6XZ3NAU7AXJ6T2X3YNE5TLN44UD5OTKHWDW7AMMH2OWUB7QQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: storage node overloaded, request limit: 2000
Dec 7 14:05:12.877232387 (46h36m0.574800703s) [noise=true , address=145.79.227.30:4763 , piece_id=TMPQE3SUM2SWP2RO3YGACVUIAVHWV2BUUX522XCSJBACMEQJT2IQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: storage node overloaded, request limit: 2000
Dec 7 14:05:12.916831066 (46h36m0.614399382s) [upload=40891]: Next returned nil segment
Dec 7 14:05:12.916839262 (46h36m0.614407567s) [upload=40891]: Waiting for error group managing segments...
Dec 7 14:05:12.929551988 (46h36m0.627120294s) [noise=true , address=145.79.227.42:4123 , piece_id=ISX3YLJJQ6L4HMTGY2J47EZM74UZJA5M65JN5TOFY6IAA2DHPABQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.929866526 (46h36m0.627434872s) [noise=true , address=145.79.227.27:4689 , piece_id=YDNSB4BUQDP7CNIAU73X6ILCKRWUYQWQEIUA2IYFPLPDGNNJ56YQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.930150587 (46h36m0.627718893s) [noise=true , address=145.79.227.43:4159 , piece_id=R7UD47XJ7XGLARQAFNL2ISONIR6VZLFJ3PWWQ5V4ZKYB7KMD6MCA, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.930672422 (46h36m0.628240757s) [noise=true , address=145.79.227.29:4941 , piece_id=ROH6OOR3PU6F6YGZMGBDVL7IZI2YFW6WOMALTBPE2DKDBH35D43Q, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.930722245 (46h36m0.628290561s) [noise=true , address=145.79.227.41:4086 , piece_id=35MLDKAKB7U26I33RKISI33JYRZK36NVREY4H43LQ2GJ7IEEW4EA, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.931139735 (46h36m0.62870804s) [noise=true , address=145.79.227.32:4834 , piece_id=PD7R4WLJLLZBCFND6BUPEJ3LIJBKNETFMLB2C2X73HH3Q4APZ6KQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.931585557 (46h36m0.629153894s) [noise=true , address=145.79.227.38:4977 , piece_id=3EAL7Y6XNXCRCNLW2SX7EADFYVVZIWQ4AJ5BQMIQV3KJKCC4C4IQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.931807582 (46h36m0.629375888s) [noise=true , address=145.79.227.23:4546 , piece_id=FTUSB2QRIKNNSIN67M5EZYRSM6MUYMC746TLBLZXTYEXLVE7MYTQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.932398866 (46h36m0.629967213s) [noise=true , address=145.79.227.40:4051 , piece_id=NS2P7FBECXVP2LCCF6UQ4BDDL7IXN4BWLRTIT6MAUOF2C5FVFI3A, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.932913437 (46h36m0.630481743s) [noise=true , address=145.79.227.43:4158 , piece_id=C7JLSPLTCF7SJVKHDYAOTLARPNSDHNU3ARV2FJQMEUR4QZOWOH3Q, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.936343114 (46h36m0.63391143s) [noise=true , address=145.79.227.31:4797 , piece_id=6NHGY624TXJ2NL4KGLCKR5TETBSH6R4KF23O3RG3LYR7OOKJ55ZA, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.938164697 (46h36m0.635733033s) [noise=true , address=45.91.116.11:4117 , piece_id=WELLJ6VZNKBHBNFTK7SZZ72J2EKFBVA6SCB27HX7JIMQUYXPY5BA, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:42.780287323 (46h36m30.477855639s) [noise=true , address=77.90.31.17:4772 , piece_id=5C5RYNJHJOR74O6MNAGKIAHGBZLGT7ZYAPHG4CJI655GS2UFFGCQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: expected piece hash; context deadline exceeded; EOF
Dec 7 14:05:42.780406927 (46h36m30.477975233s) [noise=true , address=77.90.31.12:4592 , piece_id=WHMBDMM2OKNWDRBIUYQG4N4S4BIHPSA3C2TNAEJ7BGRMDK5G726A, seg_pos={0 0}, upload=40891]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF
Dec 7 14:05:42.780455096 (46h36m30.478023402s) [seg_pos={0 0}, upload=40891]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec 7 14:05:42.813667787 (46h36m30.511236093s) [upload=40891]: Done uploading segments...
Dec 7 14:05:42.813692484 (46h36m30.5112608s) [upload=40891]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 342.284746ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
466.102681ms trace:
https://jaeger.dev.storj.io/trace/5ee917a62999ce3b
FAIL (2025-12-07 14:05:56 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec 7 14:05:12.793746279 (46h36m0.491314615s) [upload=40891]: Starting upload
Dec 7 14:05:12.793820377 (46h36m0.491388713s) [upload=40891]: Done starting upload
Dec 7 14:05:12.794040619 (46h36m0.491608935s) [upload=40891]: Uploading segments...
Dec 7 14:05:12.794207350 (46h36m0.491775685s) [upload=40891]: Got next segment. Inline: false
Dec 7 14:05:12.794270879 (46h36m0.491839214s) [upload=40891]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 7 14:05:12.794277000 (46h36m0.491845346s) [upload=40891]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 7 14:05:12.815883299 (46h36m0.513451636s) [seg_pos={0 0}, upload=40891]: Begin upload segment...
Dec 7 14:05:12.815956126 (46h36m0.513524472s) [seg_pos={0 0}, upload=40891]: Done begin upload segment.
Dec 7 14:05:12.815973168 (46h36m0.513541474s) [noise=true , address=145.79.227.29:4941 , piece_id=ROH6OOR3PU6F6YGZMGBDVL7IZI2YFW6WOMALTBPE2DKDBH35D43Q, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.815992825 (46h36m0.513561161s) [noise=true , address=77.90.31.17:4772 , piece_id=5C5RYNJHJOR74O6MNAGKIAHGBZLGT7ZYAPHG4CJI655GS2UFFGCQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.816593907 (46h36m0.514162243s) [noise=true , address=145.79.227.42:4123 , piece_id=ISX3YLJJQ6L4HMTGY2J47EZM74UZJA5M65JN5TOFY6IAA2DHPABQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.817033549 (46h36m0.514601884s) [noise=true , address=145.79.227.32:4834 , piece_id=PD7R4WLJLLZBCFND6BUPEJ3LIJBKNETFMLB2C2X73HH3Q4APZ6KQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.817608943 (46h36m0.515177279s) [noise=true , address=145.79.227.30:4763 , piece_id=TMPQE3SUM2SWP2RO3YGACVUIAVHWV2BUUX522XCSJBACMEQJT2IQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.818051330 (46h36m0.515619666s) [noise=true , address=145.79.227.40:4051 , piece_id=NS2P7FBECXVP2LCCF6UQ4BDDL7IXN4BWLRTIT6MAUOF2C5FVFI3A, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.818512791 (46h36m0.516081127s) [noise=true , address=145.79.227.41:4086 , piece_id=35MLDKAKB7U26I33RKISI33JYRZK36NVREY4H43LQ2GJ7IEEW4EA, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.819185398 (46h36m0.516753734s) [noise=true , address=145.79.227.23:4546 , piece_id=FTUSB2QRIKNNSIN67M5EZYRSM6MUYMC746TLBLZXTYEXLVE7MYTQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.821078003 (46h36m0.518646339s) [noise=true , address=145.79.227.43:4159 , piece_id=R7UD47XJ7XGLARQAFNL2ISONIR6VZLFJ3PWWQ5V4ZKYB7KMD6MCA, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.822854892 (46h36m0.520423238s) [noise=true , address=45.91.116.11:4117 , piece_id=WELLJ6VZNKBHBNFTK7SZZ72J2EKFBVA6SCB27HX7JIMQUYXPY5BA, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.824567812 (46h36m0.522136148s) [noise=true , address=145.79.227.31:4797 , piece_id=6NHGY624TXJ2NL4KGLCKR5TETBSH6R4KF23O3RG3LYR7OOKJ55ZA, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.826399914 (46h36m0.52396823s) [noise=true , address=145.79.227.43:4158 , piece_id=C7JLSPLTCF7SJVKHDYAOTLARPNSDHNU3ARV2FJQMEUR4QZOWOH3Q, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.827587042 (46h36m0.525155388s) [noise=true , address=145.79.227.38:4977 , piece_id=3EAL7Y6XNXCRCNLW2SX7EADFYVVZIWQ4AJ5BQMIQV3KJKCC4C4IQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.828083339 (46h36m0.525651645s) [noise=true , address=145.79.227.27:4689 , piece_id=YDNSB4BUQDP7CNIAU73X6ILCKRWUYQWQEIUA2IYFPLPDGNNJ56YQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.829168395 (46h36m0.526736702s) [noise=true , address=77.90.31.16:4734 , piece_id=EU5C6XZ3NAU7AXJ6T2X3YNE5TLN44UD5OTKHWDW7AMMH2OWUB7QQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.830278419 (46h36m0.527846735s) [noise=true , address=77.90.31.12:4592 , piece_id=WHMBDMM2OKNWDRBIUYQG4N4S4BIHPSA3C2TNAEJ7BGRMDK5G726A, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.876303152 (46h36m0.573871468s) [noise=true , address=77.90.31.16:4734 , piece_id=EU5C6XZ3NAU7AXJ6T2X3YNE5TLN44UD5OTKHWDW7AMMH2OWUB7QQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: storage node overloaded, request limit: 2000
Dec 7 14:05:12.877232387 (46h36m0.574800703s) [noise=true , address=145.79.227.30:4763 , piece_id=TMPQE3SUM2SWP2RO3YGACVUIAVHWV2BUUX522XCSJBACMEQJT2IQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: storage node overloaded, request limit: 2000
Dec 7 14:05:12.916831066 (46h36m0.614399382s) [upload=40891]: Next returned nil segment
Dec 7 14:05:12.916839262 (46h36m0.614407567s) [upload=40891]: Waiting for error group managing segments...
Dec 7 14:05:12.929551988 (46h36m0.627120294s) [noise=true , address=145.79.227.42:4123 , piece_id=ISX3YLJJQ6L4HMTGY2J47EZM74UZJA5M65JN5TOFY6IAA2DHPABQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.929866526 (46h36m0.627434872s) [noise=true , address=145.79.227.27:4689 , piece_id=YDNSB4BUQDP7CNIAU73X6ILCKRWUYQWQEIUA2IYFPLPDGNNJ56YQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.930150587 (46h36m0.627718893s) [noise=true , address=145.79.227.43:4159 , piece_id=R7UD47XJ7XGLARQAFNL2ISONIR6VZLFJ3PWWQ5V4ZKYB7KMD6MCA, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.930672422 (46h36m0.628240757s) [noise=true , address=145.79.227.29:4941 , piece_id=ROH6OOR3PU6F6YGZMGBDVL7IZI2YFW6WOMALTBPE2DKDBH35D43Q, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.930722245 (46h36m0.628290561s) [noise=true , address=145.79.227.41:4086 , piece_id=35MLDKAKB7U26I33RKISI33JYRZK36NVREY4H43LQ2GJ7IEEW4EA, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.931139735 (46h36m0.62870804s) [noise=true , address=145.79.227.32:4834 , piece_id=PD7R4WLJLLZBCFND6BUPEJ3LIJBKNETFMLB2C2X73HH3Q4APZ6KQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.931585557 (46h36m0.629153894s) [noise=true , address=145.79.227.38:4977 , piece_id=3EAL7Y6XNXCRCNLW2SX7EADFYVVZIWQ4AJ5BQMIQV3KJKCC4C4IQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.931807582 (46h36m0.629375888s) [noise=true , address=145.79.227.23:4546 , piece_id=FTUSB2QRIKNNSIN67M5EZYRSM6MUYMC746TLBLZXTYEXLVE7MYTQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.932398866 (46h36m0.629967213s) [noise=true , address=145.79.227.40:4051 , piece_id=NS2P7FBECXVP2LCCF6UQ4BDDL7IXN4BWLRTIT6MAUOF2C5FVFI3A, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.932913437 (46h36m0.630481743s) [noise=true , address=145.79.227.43:4158 , piece_id=C7JLSPLTCF7SJVKHDYAOTLARPNSDHNU3ARV2FJQMEUR4QZOWOH3Q, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.936343114 (46h36m0.63391143s) [noise=true , address=145.79.227.31:4797 , piece_id=6NHGY624TXJ2NL4KGLCKR5TETBSH6R4KF23O3RG3LYR7OOKJ55ZA, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.938164697 (46h36m0.635733033s) [noise=true , address=45.91.116.11:4117 , piece_id=WELLJ6VZNKBHBNFTK7SZZ72J2EKFBVA6SCB27HX7JIMQUYXPY5BA, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:42.780287323 (46h36m30.477855639s) [noise=true , address=77.90.31.17:4772 , piece_id=5C5RYNJHJOR74O6MNAGKIAHGBZLGT7ZYAPHG4CJI655GS2UFFGCQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: expected piece hash; context deadline exceeded; EOF
Dec 7 14:05:42.780406927 (46h36m30.477975233s) [noise=true , address=77.90.31.12:4592 , piece_id=WHMBDMM2OKNWDRBIUYQG4N4S4BIHPSA3C2TNAEJ7BGRMDK5G726A, seg_pos={0 0}, upload=40891]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF
Dec 7 14:05:42.780455096 (46h36m30.478023402s) [seg_pos={0 0}, upload=40891]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec 7 14:05:42.813667787 (46h36m30.511236093s) [upload=40891]: Done uploading segments...
Dec 7 14:05:42.813692484 (46h36m30.5112608s) [upload=40891]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 466.118783ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.034823074s trace:
https://jaeger.dev.storj.io/trace/5ee917a62999ce3b
FAIL (2025-12-07 14:05:42 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec 7 14:05:12.793746279 (46h36m0.491314615s) [upload=40891]: Starting upload
Dec 7 14:05:12.793820377 (46h36m0.491388713s) [upload=40891]: Done starting upload
Dec 7 14:05:12.794040619 (46h36m0.491608935s) [upload=40891]: Uploading segments...
Dec 7 14:05:12.794207350 (46h36m0.491775685s) [upload=40891]: Got next segment. Inline: false
Dec 7 14:05:12.794270879 (46h36m0.491839214s) [upload=40891]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 7 14:05:12.794277000 (46h36m0.491845346s) [upload=40891]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 7 14:05:12.815883299 (46h36m0.513451636s) [seg_pos={0 0}, upload=40891]: Begin upload segment...
Dec 7 14:05:12.815956126 (46h36m0.513524472s) [seg_pos={0 0}, upload=40891]: Done begin upload segment.
Dec 7 14:05:12.815973168 (46h36m0.513541474s) [noise=true , address=145.79.227.29:4941 , piece_id=ROH6OOR3PU6F6YGZMGBDVL7IZI2YFW6WOMALTBPE2DKDBH35D43Q, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.815992825 (46h36m0.513561161s) [noise=true , address=77.90.31.17:4772 , piece_id=5C5RYNJHJOR74O6MNAGKIAHGBZLGT7ZYAPHG4CJI655GS2UFFGCQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.816593907 (46h36m0.514162243s) [noise=true , address=145.79.227.42:4123 , piece_id=ISX3YLJJQ6L4HMTGY2J47EZM74UZJA5M65JN5TOFY6IAA2DHPABQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.817033549 (46h36m0.514601884s) [noise=true , address=145.79.227.32:4834 , piece_id=PD7R4WLJLLZBCFND6BUPEJ3LIJBKNETFMLB2C2X73HH3Q4APZ6KQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.817608943 (46h36m0.515177279s) [noise=true , address=145.79.227.30:4763 , piece_id=TMPQE3SUM2SWP2RO3YGACVUIAVHWV2BUUX522XCSJBACMEQJT2IQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.818051330 (46h36m0.515619666s) [noise=true , address=145.79.227.40:4051 , piece_id=NS2P7FBECXVP2LCCF6UQ4BDDL7IXN4BWLRTIT6MAUOF2C5FVFI3A, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.818512791 (46h36m0.516081127s) [noise=true , address=145.79.227.41:4086 , piece_id=35MLDKAKB7U26I33RKISI33JYRZK36NVREY4H43LQ2GJ7IEEW4EA, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.819185398 (46h36m0.516753734s) [noise=true , address=145.79.227.23:4546 , piece_id=FTUSB2QRIKNNSIN67M5EZYRSM6MUYMC746TLBLZXTYEXLVE7MYTQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.821078003 (46h36m0.518646339s) [noise=true , address=145.79.227.43:4159 , piece_id=R7UD47XJ7XGLARQAFNL2ISONIR6VZLFJ3PWWQ5V4ZKYB7KMD6MCA, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.822854892 (46h36m0.520423238s) [noise=true , address=45.91.116.11:4117 , piece_id=WELLJ6VZNKBHBNFTK7SZZ72J2EKFBVA6SCB27HX7JIMQUYXPY5BA, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.824567812 (46h36m0.522136148s) [noise=true , address=145.79.227.31:4797 , piece_id=6NHGY624TXJ2NL4KGLCKR5TETBSH6R4KF23O3RG3LYR7OOKJ55ZA, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.826399914 (46h36m0.52396823s) [noise=true , address=145.79.227.43:4158 , piece_id=C7JLSPLTCF7SJVKHDYAOTLARPNSDHNU3ARV2FJQMEUR4QZOWOH3Q, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.827587042 (46h36m0.525155388s) [noise=true , address=145.79.227.38:4977 , piece_id=3EAL7Y6XNXCRCNLW2SX7EADFYVVZIWQ4AJ5BQMIQV3KJKCC4C4IQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.828083339 (46h36m0.525651645s) [noise=true , address=145.79.227.27:4689 , piece_id=YDNSB4BUQDP7CNIAU73X6ILCKRWUYQWQEIUA2IYFPLPDGNNJ56YQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.829168395 (46h36m0.526736702s) [noise=true , address=77.90.31.16:4734 , piece_id=EU5C6XZ3NAU7AXJ6T2X3YNE5TLN44UD5OTKHWDW7AMMH2OWUB7QQ, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.830278419 (46h36m0.527846735s) [noise=true , address=77.90.31.12:4592 , piece_id=WHMBDMM2OKNWDRBIUYQG4N4S4BIHPSA3C2TNAEJ7BGRMDK5G726A, seg_pos={0 0}, upload=40891]: Uploading piece...
Dec 7 14:05:12.876303152 (46h36m0.573871468s) [noise=true , address=77.90.31.16:4734 , piece_id=EU5C6XZ3NAU7AXJ6T2X3YNE5TLN44UD5OTKHWDW7AMMH2OWUB7QQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: storage node overloaded, request limit: 2000
Dec 7 14:05:12.877232387 (46h36m0.574800703s) [noise=true , address=145.79.227.30:4763 , piece_id=TMPQE3SUM2SWP2RO3YGACVUIAVHWV2BUUX522XCSJBACMEQJT2IQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: storage node overloaded, request limit: 2000
Dec 7 14:05:12.916831066 (46h36m0.614399382s) [upload=40891]: Next returned nil segment
Dec 7 14:05:12.916839262 (46h36m0.614407567s) [upload=40891]: Waiting for error group managing segments...
Dec 7 14:05:12.929551988 (46h36m0.627120294s) [noise=true , address=145.79.227.42:4123 , piece_id=ISX3YLJJQ6L4HMTGY2J47EZM74UZJA5M65JN5TOFY6IAA2DHPABQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.929866526 (46h36m0.627434872s) [noise=true , address=145.79.227.27:4689 , piece_id=YDNSB4BUQDP7CNIAU73X6ILCKRWUYQWQEIUA2IYFPLPDGNNJ56YQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.930150587 (46h36m0.627718893s) [noise=true , address=145.79.227.43:4159 , piece_id=R7UD47XJ7XGLARQAFNL2ISONIR6VZLFJ3PWWQ5V4ZKYB7KMD6MCA, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.930672422 (46h36m0.628240757s) [noise=true , address=145.79.227.29:4941 , piece_id=ROH6OOR3PU6F6YGZMGBDVL7IZI2YFW6WOMALTBPE2DKDBH35D43Q, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.930722245 (46h36m0.628290561s) [noise=true , address=145.79.227.41:4086 , piece_id=35MLDKAKB7U26I33RKISI33JYRZK36NVREY4H43LQ2GJ7IEEW4EA, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.931139735 (46h36m0.62870804s) [noise=true , address=145.79.227.32:4834 , piece_id=PD7R4WLJLLZBCFND6BUPEJ3LIJBKNETFMLB2C2X73HH3Q4APZ6KQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.931585557 (46h36m0.629153894s) [noise=true , address=145.79.227.38:4977 , piece_id=3EAL7Y6XNXCRCNLW2SX7EADFYVVZIWQ4AJ5BQMIQV3KJKCC4C4IQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.931807582 (46h36m0.629375888s) [noise=true , address=145.79.227.23:4546 , piece_id=FTUSB2QRIKNNSIN67M5EZYRSM6MUYMC746TLBLZXTYEXLVE7MYTQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.932398866 (46h36m0.629967213s) [noise=true , address=145.79.227.40:4051 , piece_id=NS2P7FBECXVP2LCCF6UQ4BDDL7IXN4BWLRTIT6MAUOF2C5FVFI3A, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.932913437 (46h36m0.630481743s) [noise=true , address=145.79.227.43:4158 , piece_id=C7JLSPLTCF7SJVKHDYAOTLARPNSDHNU3ARV2FJQMEUR4QZOWOH3Q, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.936343114 (46h36m0.63391143s) [noise=true , address=145.79.227.31:4797 , piece_id=6NHGY624TXJ2NL4KGLCKR5TETBSH6R4KF23O3RG3LYR7OOKJ55ZA, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:12.938164697 (46h36m0.635733033s) [noise=true , address=45.91.116.11:4117 , piece_id=WELLJ6VZNKBHBNFTK7SZZ72J2EKFBVA6SCB27HX7JIMQUYXPY5BA, seg_pos={0 0}, upload=40891]: Done uploading piece. err: <nil>
Dec 7 14:05:42.780287323 (46h36m30.477855639s) [noise=true , address=77.90.31.17:4772 , piece_id=5C5RYNJHJOR74O6MNAGKIAHGBZLGT7ZYAPHG4CJI655GS2UFFGCQ, seg_pos={0 0}, upload=40891]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: expected piece hash; context deadline exceeded; EOF
Dec 7 14:05:42.780406927 (46h36m30.477975233s) [noise=true , address=77.90.31.12:4592 , piece_id=WHMBDMM2OKNWDRBIUYQG4N4S4BIHPSA3C2TNAEJ7BGRMDK5G726A, seg_pos={0 0}, upload=40891]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF
Dec 7 14:05:42.780455096 (46h36m30.478023402s) [seg_pos={0 0}, upload=40891]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec 7 14:05:42.813667787 (46h36m30.511236093s) [upload=40891]: Done uploading segments...
Dec 7 14:05:42.813692484 (46h36m30.5112608s) [upload=40891]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 30.034851147s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
374.320183ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 09:14:13 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 374.346914ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
324.637696ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 09:13:28 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 324.65588ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
325.298792ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 09:12:14 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 325.317416ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
346.136928ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 09:11:55 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 346.176652ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
330.386254ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 09:11:36 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 330.402925ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
462.758868ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 09:10:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 462.77599ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
342.607886ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 09:09:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 342.651158ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
373.08027ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 09:08:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 373.100056ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
339.575893ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 09:08:02 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 339.604898ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
443.149861ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 09:07:13 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 443.167163ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
322.023873ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 09:06:20 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 322.043579ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
341.152518ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 09:05:25 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 341.169629ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
349.510138ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 09:05:02 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 349.534205ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
362.554795ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 09:04:09 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 362.575785ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
372.343302ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 09:03:06 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 372.368489ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
331.942311ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 09:02:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 331.954162ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
405.063385ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 09:01:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 405.082811ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
325.227017ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 09:01:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 325.251474ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
327.529652ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 09:00:12 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 327.54456ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
368.290112ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 08:59:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 368.306933ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
320.215378ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 08:58:24 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 320.2327ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
337.852531ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 08:57:17 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 337.869032ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
369.304696ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 08:56:50 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 369.351915ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
357.088935ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 08:55:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 357.111428ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
395.571894ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 08:54:56 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 395.592614ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
326.906082ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 08:53:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 326.922835ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
590.030542ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 08:52:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 590.07237ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
356.433973ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 08:51:28 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 356.454812ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
349.509989ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 08:51:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 349.523445ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
536.934683ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 08:50:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 536.950833ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
379.191878ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 08:49:22 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 379.230771ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
324.219126ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 08:48:07 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 324.236718ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
423.001566ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 08:47:28 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 423.021012ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
375.109673ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 08:47:03 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 375.121475ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
402.862534ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 08:46:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 402.878935ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
342.830431ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 08:45:28 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 342.848265ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
335.986709ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 08:44:35 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 336.005874ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
385.642577ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 08:43:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 385.661752ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
385.66467ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 08:42:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 385.686841ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
376.419931ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 08:41:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 376.44161ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
340.289968ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 08:41:42 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 340.328178ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
352.574374ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 08:40:47 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 352.594371ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
376.138033ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 08:40:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 376.15251ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
386.268868ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 08:39:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 386.280379ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
348.888263ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 08:38:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 348.904984ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
373.810365ms trace:
https://jaeger.dev.storj.io/trace/6295afa6d8d7f95
FAIL (2025-12-06 08:37:49 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 373.858474ms
edge
gateway
us-select-1
|