Only Coders - Where knowledge meets opportunity

python (65.2k questions)

javascript (44.3k questions)

reactjs (22.7k questions)

java (20.8k questions)

c# (17.4k questions)

html (16.3k questions)

r (13.7k questions)

android (13k questions)

Questions - language-model

What is the correct return of BertForMaskedLM?

I'm using huggingface BertForMaskedLM. For a sentence, I'm getting a 3-dimensional return from BertForMaskedLM. For example (P,N,V), Here I understand the N is the length of the sentence and V is the ...

kowser66

nlp

huggingface-transformers

bert-language-model

transformer-model

language-model

Votes: 0

Answers: 1

Latest Answer

Your answer most probably lies in the documentation. I linked the MaskedLMOutput which is what BertForMaskedLM forward method returns.

Manuel Faysse

How bert is a bidirectional?

Bert encoder takes the input and goes for the multi-head attention model. But how do they maintain sequence? Since current words don't take sequence of previous words. Besides, why is it bidirectional...

kowser66

nlp

lstm

bert-language-model

language-model

bilstm

Votes: 0

Answers: 1

Latest Answer

It is bidirectional because it uses context from both sides of the current word (instead of e.g. using just the previous few words it uses the whole sequence). It depends on how much you want to go in...

ewz93

ValueError: Layer weight shape (30522, 768) not compatible with provided weight shape ()

I got word-embedding using BERT and need to feed it as an embedding layer in the Keras model, and the error I got is ValueError: Layer weight shape (30522, 768) not compatible with provided weight sha...

user3188912

keras

huggingface-transformers

bert-language-model

transformer-model

language-model

Votes: 0

Answers: 1

Latest Answer

You are passing to set_weights a list of list: embedding_matrix = [np.random.uniform(0,1, (30522, 768))] sentence = Input((20,)) embedding = Embedding(30522, 768, mask_zero=True)(sentence) model = Mo...

Marco Cerliani

Posts

Questions

Blogs

Jobs