Here, I give an liner operator perspective for LTI systems, which will be instrumental for rigorously discussing the duality between controllability-observability and LQR-Kalman filters beyond taking transposes!
Consider the stable LTI system (with $A$ Hurwitz)
$$ \dot x(t) = A x(t) + B u(t), \qquad y(t) = C x(t) + D u(t), \qquad x(0)=0. $$
On a finite horizon $[0,T]$, we treat inputs/outputs as elements of the Hilbert space $L_2[0,T]$ with inner product
$$ \langle f,g\rangle_{[0,T]} \;=\; \int_0^T g(t)^\top f(t)\,dt. $$
Define the linear operator that represents the input–output relationship:
$$ \mathbf{G} : L_2[0,T] \to L_2[0,T], \qquad y = \mathbf Gu, $$
given by convolution
$$ (\mathbf{G}u)(t) = \int_0^t G(t-s)u(s)\,ds, $$
with impulse response
$$ G(t)= \begin{cases} C e^{At} B + D\,\delta(t), & t\ge 0,\\ 0, & t<0. \end{cases} $$
This is the transfer function matrix (in time domain) that captures the input-output relation, i.e., $y(t)=(\mathbf{G}u)(t)$. We can get the standard transfer function matrix in the frequency domain via the Laplace transform!
For the linear operator $\mathbf{G}$, the adjoint $\mathbf{G}^*$ is defined by
$$ \langle \mathbf{G}u,\eta\rangle_{[0,T]} \;=\; \langle u, \mathbf{G}^*\eta\rangle_{[0,T]} \qquad \forall\,u,\eta\in L_2[0,T]. $$
We can compute this for $G$. A direct calculation (see the bottom of this page for the details) yields
$$ (\mathbf{G}^*\eta)(s) = \int_s^T G^\top(t-s)\,\eta(t)\,dt. $$
In state-space form, $G^*$ corresponds to the backward-time adjoint dynamics