It's a system engineering idea:
- Encapsulate a function,
- Make it opaque to outsiders, and
- Feed it inputs and external controls, biases, and offsets
- Look for and utilize the prescribed outcomes
- Voila! You've got a black box!
Ooops!
What if ...
What if there are chaotic responses, unforeseen resonances, inexplicable outcomes and distortions, and even destructive behavior?
The question is begged: Do you understand how your black box works? Very likely if you don't know, the larger system doesn't either because, hey!, you designed the larger system!
If your answer is No to the above, then here's some advice: roll back the black box and expose the functionality to the point you can understand it; you can defend it; you can reliably forecast it's role in your system
Working on AI?
The modern idea of AI with multiple layered meshes of data processors may be the ultimate black box. If this is your domain, read a bit of this Wired article to get onto my point of view.
Buy them at any online book retailer!
