If you want to do it without purchasing any additional software, format the disk as exFAT format. Windows and macOS can natively read and write ExFAT disks and it can hand files larger than 4 GB.
FAT32 is an older format that is cross-platform compatible with nearly every operating system on earth, but can’t handle files larger than 4 GB and can’t handle a partition larger than 2 TB.
NTFS is another option as it’s natively supported on Windows but you will need to purchase 3rd party software such as Microsoft NTFS for Mac by Paragon Software in order to have full read/write access on the Mac. Without the Paragon software, you’ll be able to read the disk on macOS, but not write to it.
Some USB hard drives come with a customised, free version of Paragon NTFS software, however the software is usually restricted to only work with the particular brand of external hard drive you have purchased.
If you have Boot Camp installed on a Mac, it also comes with a read-only driver for HFS Plus so that the Windows side of things can read your Mac disk, but you can’t write files back to it.