Use the Photos iOS 14 widget on your iPhone? If so, you want to read the linked post.
Michael Potuck, 9to5Mac, walks through the process of telling the widget not to display a specific photo, and clarifies where the widget sources those photos from. Good stuff.