It is a pain in the ass.
Several Windows tools exist which add support for the file systems used in the Linux world.
- linux reader
See here for a comparison of existing tools.
However, I can not say that any of these worked properly for ext4. Not to speak of ZFS or btrfs 😉