Lernen Multiplicative and Complex Models: DistMult, ComplEx

Multiplicative and complex-valued embedding models offer a powerful way to capture the semantics of knowledge graphs, especially when dealing with a variety of relation types. Two such models, DistMult and ComplEx, are widely used for their elegant scoring mechanisms and their differing abilities to represent symmetric and antisymmetric relations.

The DistMult model represents entities and relations as real - valued vectors of the same dimension. Its scoring function for a triple ( $h$ , $r$ , $t$ ) — where $h$ is the head entity, $r$ is the relation, and $t$ is the tail entity — uses a simple elementwise multiplication:

The score is computed by taking the sum of the elementwise product of the head, relation, and tail vectors.

This can be written as:

$\text{score}(h, r, t) = \sum h \times r \times t$

Because multiplication is commutative, DistMult is inherently limited to modeling symmetric relations, meaning it cannot distinguish between ( $h$ , $r$ , $t$ ) and ( $t$ , $r$ , $h$ ).

The ComplEx model extends DistMult by introducing complex-valued embeddings. Entities and relations are represented as complex vectors, and the scoring function incorporates the complex conjugate of the tail entity:

$\text{score}(h, r, t) = \operatorname{Re}\left( \sum h \times r \times \overline{t} \right)$

Here, $\operatorname{Re}$ denotes the real part, and $\overline{t}$ is the complex conjugate. This design allows ComplEx to model both symmetric and antisymmetric relations, making it more expressive for knowledge graphs where directionality matters.

Both models are efficient and scalable, but their scoring mechanisms lead to different capabilities in representing relation patterns.


              1234567891011121314151617181920212223
            
import numpy as np

# DistMult scoring for real-valued embeddings
def distmult_score(h, r, t):
    # h, r, t: numpy arrays of shape (d,)
    return np.sum(h * r * t)

# ComplEx scoring for complex-valued embeddings
def complex_score(h, r, t):
    # h, r, t: numpy arrays of shape (d,), dtype=complex
    return np.real(np.sum(h * r * np.conj(t)))

# Example vectors
h_real = np.array([0.2, -1.0, 0.5])
r_real = np.array([1.0, 0.3, -0.7])
t_real = np.array([-0.4, 0.8, 1.2])

h_cplx = np.array([0.2 + 0.1j, -1.0 + 0.5j, 0.5 - 0.2j])
r_cplx = np.array([1.0 - 0.3j, 0.3 + 0.2j, -0.7 + 0.4j])
t_cplx = np.array([-0.4 + 0.7j, 0.8 - 0.6j, 1.2 + 0.3j])

print("DistMult score:", distmult_score(h_real, r_real, t_real))
print("ComplEx score:", complex_score(h_cplx, r_cplx, t_cplx))

Definition

A symmetric relation is one where if a triple ( $h$ , $r$ , $t$ ) holds, then ( $t$ , $r$ , $h$ ) also holds. An antisymmetric relation is one where if ( $h$ , $r$ , $t$ ) holds, then ( $t$ , $r$ , $h$ ) does not hold unless $h = t$ . Symmetry is crucial for modeling relations like "sibling of," while antisymmetry is needed for "parent of".

War alles klar?

Danke für Ihr Feedback!

Abschnitt 2. Kapitel 4

Fragen Sie AI

Fragen Sie alles oder probieren Sie eine der vorgeschlagenen Fragen, um unser Gespräch zu beginnen

Suggested prompts:

Can you explain why DistMult cannot model antisymmetric relations?

How does the use of complex numbers in ComplEx help with modeling directionality?

Can you provide more examples of symmetric and antisymmetric relations in knowledge graphs?

Swipe um das Menü anzuzeigen

The score is computed by taking the sum of the elementwise product of the head, relation, and tail vectors.