Lære Multiplicative and Complex Models: DistMult, ComplEx

Multiplicative and complex-valued embedding models offer a powerful way to capture the semantics of knowledge graphs, especially when dealing with a variety of relation types. Two such models, DistMult and ComplEx, are widely used for their elegant scoring mechanisms and their differing abilities to represent symmetric and antisymmetric relations.

The DistMult model represents entities and relations as real - valued vectors of the same dimension. Its scoring function for a triple ( $h$ , $r$ , $t$ ) — where $h$ is the head entity, $r$ is the relation, and $t$ is the tail entity — uses a simple elementwise multiplication:

The score is computed by taking the sum of the elementwise product of the head, relation, and tail vectors.

This can be written as:

$\text{score}(h, r, t) = \sum h \times r \times t$

Because multiplication is commutative, DistMult is inherently limited to modeling symmetric relations, meaning it cannot distinguish between ( $h$ , $r$ , $t$ ) and ( $t$ , $r$ , $h$ ).

The ComplEx model extends DistMult by introducing complex-valued embeddings. Entities and relations are represented as complex vectors, and the scoring function incorporates the complex conjugate of the tail entity:

$\text{score}(h, r, t) = \operatorname{Re}\left( \sum h \times r \times \overline{t} \right)$

Here, $\operatorname{Re}$ denotes the real part, and $\overline{t}$ is the complex conjugate. This design allows ComplEx to model both symmetric and antisymmetric relations, making it more expressive for knowledge graphs where directionality matters.

Both models are efficient and scalable, but their scoring mechanisms lead to different capabilities in representing relation patterns.


              1234567891011121314151617181920212223
            
import numpy as np

# DistMult scoring for real-valued embeddings
def distmult_score(h, r, t):
    # h, r, t: numpy arrays of shape (d,)
    return np.sum(h * r * t)

# ComplEx scoring for complex-valued embeddings
def complex_score(h, r, t):
    # h, r, t: numpy arrays of shape (d,), dtype=complex
    return np.real(np.sum(h * r * np.conj(t)))

# Example vectors
h_real = np.array([0.2, -1.0, 0.5])
r_real = np.array([1.0, 0.3, -0.7])
t_real = np.array([-0.4, 0.8, 1.2])

h_cplx = np.array([0.2 + 0.1j, -1.0 + 0.5j, 0.5 - 0.2j])
r_cplx = np.array([1.0 - 0.3j, 0.3 + 0.2j, -0.7 + 0.4j])
t_cplx = np.array([-0.4 + 0.7j, 0.8 - 0.6j, 1.2 + 0.3j])

print("DistMult score:", distmult_score(h_real, r_real, t_real))
print("ComplEx score:", complex_score(h_cplx, r_cplx, t_cplx))

Definition

A symmetric relation is one where if a triple ( $h$ , $r$ , $t$ ) holds, then ( $t$ , $r$ , $h$ ) also holds. An antisymmetric relation is one where if ( $h$ , $r$ , $t$ ) holds, then ( $t$ , $r$ , $h$ ) does not hold unless $h = t$ . Symmetry is crucial for modeling relations like "sibling of," while antisymmetry is needed for "parent of".

Alt var klart?

Takk for tilbakemeldingene dine!

Seksjon 2. Kapittel 4

Spør AI

Spør om hva du vil, eller prøv ett av de foreslåtte spørsmålene for å starte chatten vår

Sveip for å vise menyen

The score is computed by taking the sum of the elementwise product of the head, relation, and tail vectors.