Collective inference is a popular approach for solving tasks as knowledge graph completion within the statistical relational learning field. There are many existing solutions for this task, however, each of them is subjected to some limitation, either by restriction to only some learning settings, lacking interpretability of the model or theoretical test error bounds. We propose an approach based on cautious inference process which uses first-order rules and provides PAC-style bounds.