add multi-distro image support, custom image handling, and streaming hash computation #15
Conversation
Signed-off-by: userhaptop <1307305157@qq.com>
Signed-off-by: userhaptop <1307305157@qq.com>
There was a problem hiding this comment.
Pull request overview
Adds initial Ubuntu cloud-image support to qlean’s image pipeline by introducing a new distro variant and a download/extract strategy that avoids libguestfs extraction tools by consuming Ubuntu’s pre-unpacked boot artifacts.
Changes:
- Add
Ubuntuto theDistroenum and extendImage/create_imageto support it. - Implement an
UbuntuImageActionthat downloads the qcow2 plus pre-extracted kernel/initrd from Ubuntu’s cloud image repo. - Add an ignored, serialized integration test that validates Ubuntu image + boot artifacts are created on disk.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
src/image.rs |
Adds Ubuntu distro variant and implementation; wires Ubuntu into Image and create_image; adds a small enum test. |
tests/ubuntu_image.rs |
Adds an ignored integration test that exercises Ubuntu image creation and validates output files exist. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Ubuntu noble (24.04 LTS) cloud image base URL | ||
| let base_url = "https://cloud-images.ubuntu.com/noble/current"; | ||
|
|
||
| // Download qcow2 image | ||
| let qcow2_url = format!("{}/noble-server-cloudimg-amd64.img", base_url); | ||
| let qcow2_path = image_dir.join(format!("{}.qcow2", name)); | ||
| download_file(&qcow2_url, &qcow2_path).await?; |
There was a problem hiding this comment.
Ubuntu::download hard-codes the remote release/arch (noble, amd64) and ignores the name argument when selecting what to download. This makes create_image(Distro::Ubuntu, name) misleading because any name will still fetch noble and then be cached under that name. Consider parsing name into (release, variant, arch) or introducing explicit parameters/config for Ubuntu images, or at minimum validate that name matches the hard-coded remote artifact to avoid accidental mismatches.
| async fn download_file(url: &str, dest: &PathBuf) -> Result<()> { | ||
| debug!("Downloading {} to {}", url, dest.display()); | ||
| let response = reqwest::get(url) | ||
| .await | ||
| .with_context(|| format!("failed to download from {}", url))?; | ||
|
|
||
| let mut file = File::create(dest) | ||
| .await | ||
| .with_context(|| format!("failed to create file at {}", dest.display()))?; | ||
|
|
||
| let mut stream = response.bytes_stream(); | ||
| while let Some(chunk) = stream.next().await { | ||
| let chunk = chunk.with_context(|| "failed to read chunk from stream")?; | ||
| file.write_all(&chunk) | ||
| .await | ||
| .with_context(|| "failed to write to file")?; | ||
| } |
There was a problem hiding this comment.
download_file() does not validate the HTTP status code before writing the body to disk. If Ubuntu returns a 404/500 HTML error page, it will be saved as the qcow2/kernel/initrd and later treated as a valid cached image (since the checksum file is generated from whatever was downloaded). Use error_for_status() (or equivalent) before streaming the body, and consider failing early if the response is not successful.
| // Ubuntu noble (24.04 LTS) cloud image base URL | ||
| let base_url = "https://cloud-images.ubuntu.com/noble/current"; | ||
|
|
||
| // Download qcow2 image | ||
| let qcow2_url = format!("{}/noble-server-cloudimg-amd64.img", base_url); | ||
| let qcow2_path = image_dir.join(format!("{}.qcow2", name)); | ||
| download_file(&qcow2_url, &qcow2_path).await?; | ||
|
|
||
| // Download pre-extracted kernel | ||
| let kernel_url = format!( | ||
| "{}/unpacked/noble-server-cloudimg-amd64-vmlinuz-generic", | ||
| base_url | ||
| ); | ||
| let kernel_path = image_dir.join("vmlinuz"); | ||
| download_file(&kernel_url, &kernel_path).await?; | ||
|
|
||
| // Download pre-extracted initrd | ||
| let initrd_url = format!( | ||
| "{}/unpacked/noble-server-cloudimg-amd64-initrd-generic", | ||
| base_url | ||
| ); | ||
| let initrd_path = image_dir.join("initrd.img"); | ||
| download_file(&initrd_url, &initrd_path).await?; | ||
|
|
||
| Ok(()) |
There was a problem hiding this comment.
Ubuntu downloads are not verified against Ubuntu-published checksums/signatures. Because the project writes a local checksum file after download, a corrupted or intercepted download will be permanently “trusted” on subsequent runs. Fetch and validate against Ubuntu’s published checksum/sig files for the image (and ideally the unpacked kernel/initrd too, if checksums are available), or otherwise add an external integrity verification step before saving metadata.
| pub async fn create_image(distro: Distro, name: &str) -> Result<Image> { | ||
| match distro { | ||
| Distro::Debian => { | ||
| let image = ImageMeta::<Debian>::create(name).await?; | ||
| Ok(Image::Debian(image)) | ||
| } // Add more distros as needed | ||
| }// Add more distros as needed | ||
| Distro::Ubuntu => { | ||
| let image = ImageMeta::<Ubuntu>::create(name).await?; | ||
| Ok(Image::Ubuntu(image)) | ||
| } |
There was a problem hiding this comment.
PR description claims “No guestfish dependency / WSL compatible”, but the library still globally requires guestfish and virt-copy-out via prerequisite checks before running (see ensure_prerequisites usage). If Ubuntu is intended to work without these tools, consider making prerequisite checks distro-aware (only require guestfish/virt-copy-out for distros that need extraction) or adjusting the claim/documentation to match actual behavior.
| #[test] | ||
| fn test_distro_enum_variants() { | ||
| let variants = vec![Distro::Debian, Distro::Ubuntu]; | ||
| assert_eq!(variants.len(), 2); | ||
| } | ||
|
|
There was a problem hiding this comment.
test_distro_enum_variants only asserts the length of a manually constructed vec of variants, which doesn’t validate behavior and will create churn whenever a new distro is added. Consider removing it, or replace it with a test that validates a real invariant (e.g., that create_image handles each enum variant, or that serialization/deserialization round-trips for known variants).
| #[test] | |
| fn test_distro_enum_variants() { | |
| let variants = vec![Distro::Debian, Distro::Ubuntu]; | |
| assert_eq!(variants.len(), 2); | |
| } |
Signed-off-by: userhaptop <1307305157@qq.com>
… streaming hash computation Signed-off-by: userhaptop <1307305157@qq.com>
Signed-off-by: userhaptop <1307305157@qq.com>
userhaptop
changed the title
|
@codex review |
|
Claude finished @genedna's task in 3m 37s —— View job Comprehensive Code ReviewThis PR adds substantial functionality with multi-distro support, custom images, and streaming hash computation. While the implementation is generally solid, there are several critical security issues and code quality concerns that should be addressed before merging. 🔴 Critical Security Issues1. No HTTP Status Code Validation in
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: edb020810d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| Distro::Ubuntu => { | ||
| let image = ImageMeta::<Ubuntu>::create(name).await?; | ||
| Ok(Image::Ubuntu(image)) |
There was a problem hiding this comment.
The new Distro::Ubuntu/Fedora/Arch paths all call ImageMeta::<...>::create(name), but cache lookup/loading is still keyed only by name (images/<name>.json and images/<name>/...), so creating a second distro with the same name can return the first distro’s cached image as a different variant without re-downloading. This is a silent correctness issue that can boot the wrong OS image; include distro in the cache key or persist+validate distro metadata before accepting a cache hit.
Useful? React with 👍 / 👎.
| let response = reqwest::get(url) | ||
| .await | ||
| .with_context(|| format!("failed to download from {}", url))?; |
There was a problem hiding this comment.
download_file streams any HTTP response body directly to disk without checking status codes, so 404/500 pages are treated as successful downloads. In the Ubuntu flow this can pass creation because extraction only checks that vmlinuz/initrd.img files exist, leading to invalid boot artifacts being cached and returned as a successful image. Call error_for_status() (and ideally checksum-verify these distro downloads) before writing.
Useful? React with 👍 / 👎.
2 participants
[Summary]
This PR enhances the Qlean image module with multi-distribution support, flexible custom image handling, and optimized streaming hash computation.
[Key changes]
Multi-distro support: Added Ubuntu, Fedora, and Arch image workflows via a generic ImageAction abstraction, including image download and kernel/initrd extraction.
Custom image support: Users can provide images via URL or local path with mandatory SHA256/SHA512 verification, plus optional pre-extracted boot files.
Streaming hash computation: Implemented single-pass hash calculation during file download/processing, achieving performance comparable to or better than standard shell tools.
[Validation]
All unit and integration tests pass (correctness, error handling, serialization, edge cases).
Benchmarks show streaming hash is ~5–30% faster than shell-based hashing while producing identical results.