Data-Efficient Learning with Neural Programs
Many computational tasks can be naturally expressed as a composition of a DNN followed by a program written in a traditional programming language or an API call to an LLM. We call such composites "neural programs" and focus on the problem of learning the DNN parameters when the training da...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Journal Article |
| Language: | English |
| Published: |
10-06-2024
|
| Subjects: | |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | Many computational tasks can be naturally expressed as a composition of a DNN
followed by a program written in a traditional programming language or an API
call to an LLM. We call such composites "neural programs" and focus on the
problem of learning the DNN parameters when the training data consist of
end-to-end input-output labels for the composite. When the program is written
in a differentiable logic programming language, techniques from neurosymbolic
learning are applicable, but in general, the learning for neural programs
requires estimating the gradients of black-box components. We present an
algorithm for learning neural programs, called ISED, that only relies on
input-output samples of black-box components. For evaluation, we introduce new
benchmarks that involve calls to modern LLMs such as GPT-4 and also consider
benchmarks from the neurosymbolic learning literature. Our evaluation shows
that for the latter benchmarks, ISED has comparable performance to
state-of-the-art neurosymbolic frameworks. For the former, we use adaptations
of prior work on gradient approximations of black-box components as a baseline,
and show that ISED achieves comparable accuracy but in a more data- and
sample-efficient manner. |
|---|---|
| DOI: | 10.48550/arxiv.2406.06246 |