Die Präsentation wird geladen. Bitte warten

Die Präsentation wird geladen. Bitte warten

PRE-ORDERING DEPENDENCY SUBTREES FOR PHRASE-BASED SMT Intern: Arianna Bisazza. Mentors: Alex Ceausu, John Tinsley.

Ähnliche Präsentationen


Präsentation zum Thema: "PRE-ORDERING DEPENDENCY SUBTREES FOR PHRASE-BASED SMT Intern: Arianna Bisazza. Mentors: Alex Ceausu, John Tinsley."—  Präsentation transkript:

1 PRE-ORDERING DEPENDENCY SUBTREES FOR PHRASE-BASED SMT Intern: Arianna Bisazza. Mentors: Alex Ceausu, John Tinsley

2 Dependency subtree pre-ordering What if… we cant/dont want to change the decoding process we have dependency parses available …one way to go: pre-order input parse trees, then translate normally Main research problems: how to pre-order? (ordering model) and what? (rule selection)

3 Dependency subtree pre-ordering Die Budapester Staat anwaltschaft hat ihre Ermittlungen zum Vorfall eingeleitet. the Budapest Prosecutors Office has its investigation on the accident initiated ihre |PPOSAT zum |APPRART Vorfall |NN hat |VAFIN anwaltschaft |NN Staat |NN Budapester |NN die |ART eingeleitet |VVPP. |$. Ermittlungen |NN NK SB OC PUNC OA NK MNR NK NN VAFIN VVPP$. NN VAFIN VVPP$. NN VVPP NN VVPP... Permute subtrees (a node + its children) Each subtree processed independently

4 Dependency subtree pre-ordering Die Budapester Staat anwaltschaft hat ihre Ermittlungen zum Vorfall eingeleitet. the Budapest Prosecutors Office has its investigation on the accident initiated ihre |PPOSAT zum |APPRART Vorfall |NN hat |VAFIN anwaltschaft |NN Staat |NN Budapester |NN die |ART eingeleitet |VVPP. |$. Ermittlungen |NN NK SB OC PUNC OA NK MNR NK NN VAFIN VVPP$. NN VAFIN VVPP$. NN VVPP NN VVPP... Permute subtrees (a node + its children) Each subtree processed independently

5 Pre-ordering model (1) – MLE Baseline model: max likelihood MLE (relative frequency-based) Subtree representation: relation type and POS tag OA|NN _OC|VVPP OA|NN _OC|VVPP Prob=0.75 Prob=0.25 OA|NN _OC|VVPP Limitations: - ambiguity due to coarse word classification (only few relation/POS tags) - coverage: many unseen or low-counts subtrees

6 Pre-ordering model (2) – SMT Idea: learn to reorder by SMT! Train a phrase-based system on pairs of original/pre-ordered source language node sequences (subtrees) Advantages: generalization: all node sequences can be processed model flexibility: represent different features as factors tune different model weights by MERT ORIGINAL SB|NN _ROOT|VAFIN OC|VVPP PUNC|$. NK|ART NK|NN NK|NN _SB|NN OA|NN _OC|VVPP... OA|NN _OC|VVPP... ORIGINAL SB|NN _ROOT|VAFIN OC|VVPP PUNC|$. NK|ART NK|NN NK|NN _SB|NN OA|NN _OC|VVPP... OA|NN _OC|VVPP... PRE-ORDERED SB|NN _ROOT|VAFIN OC|VVPP PUNC|$. NK|ART NK|NN NK|NN _SB|NN _OC|VVPP OA|NN... OA|NN _OC|VVPP... PRE-ORDERED SB|NN _ROOT|VAFIN OC|VVPP PUNC|$. NK|ART NK|NN NK|NN _SB|NN _OC|VVPP OA|NN... OA|NN _OC|VVPP...

7 Pre-ordering model (2) – SMT Possible models: original-to-preordered phrase table target (preordered) n-gram language models lexicalized reordering models at the level of relation type, POS tags or words etc. all models log-linearly combined weights tuned by MERT, optimizing reo.score (KRS) ORIGINAL SB|NN| anwaltschaft _ROOT|VAFIN| hat OC|VVPP| eingeleitet PUNC|$.|. NK|ART| die NK|NN| Budapester NK|NN| Staat _SB|NN| anwaltschaft OA|NN| Ermittlungen _OC|VVPP| eingeleitet... ORIGINAL SB|NN| anwaltschaft _ROOT|VAFIN| hat OC|VVPP| eingeleitet PUNC|$.|. NK|ART| die NK|NN| Budapester NK|NN| Staat _SB|NN| anwaltschaft OA|NN| Ermittlungen _OC|VVPP| eingeleitet... Each feature type is represented as a factor, for example:

8 Evaluation Training/dev/test: 495/2.5/2.5K sent. from WMT-12 De-En train data 1.6M/8K/9K training subtrees (rooted at verb nodes) MethodAdd. modelsBLEUKRSACCUNK MLE-rel MLE-relPOS SMT-relPOS (moses) LM(rel) +LM(POS) lexreo(relPOS) lexreo(words) lexreo(nodeSpan) MethodAdd. modelsBLEUKRSACCUNK MLE-relPOS SMT-relPOS (moses) +lexreo(relPOS) M/23K/24K training subtrees (all with >1 node)

9 Selective pre-ordering Not all subtrees need to be pre-ordered (especially in language pairs like German-English) How to select them? Approach: compute average distortion gain on training data, then only pre-order subtrees with high distortion gain Pre-ordering performances, with two different thresholds Selection%subtreesMethodAdd. modelsBLEUKRSACCUNK None (all subtrees) 100% MLE SMT +lexreo(relPOS) HRPd15f313% MLE SMT +lexreo(relPOS) SRPd20f33% MLE SMT +lexreo(relPOS)

10 MT experiments Using WMT-12 De-En training and test data InputDL newstest2009newstest2010 BLEUKRSBLEUKRS Original Reo.all Reo.verbRoot Reo.HRPd15f Reo.SRPd20f Original Reo.all Reo.verbRoot Reo.HRPd15f Reo.SRPd20f

11 MT output examples (1) ORI: nach dem steilen Abfall am Morgen konnte die Prager Börse die Verluste korrigieren. REO: nach dem steilen Abfall am Morgen die Prager Börse konnte die Verluste korrigieren. REF: after a sharp drop in the morning, the Prague Stock Market corrected its losses. BASE: after the sharp falls on the morning, the Prague Stock Exchange to correct the losses. NEW: after the sharp falls on the morning the Prague Stock Exchange was able to correct the losses.

12 MT output examples (2) ORI: … über einen Plan, der funktionieren wird und der auf dem Markt auch wirksam sein muss. REO: … über einen Plan, der wird funktionieren und der muss sein auch wirksam auf dem Markt. REF: … on a plan which will function and which also must be effective on the market. BASE: … on a plan that will work and on the market also needs to be effective. NEW: … on a plan that will work and must also be effective on the market.

13 MT output examples (3) ORI: die Kongress Abgeordneten müssen nämlich noch einige Details der Vereinbarung aushandeln, ehe sie die Endfassung des Gesetzes veröffentlichen und darüber abstimmen dürfen. REO: die Kongress Abgeordneten müssen nämlich aushandeln, ehe sie veröffentlichen die Endfassung des Gesetzes und dürfen darüber abstimmen noch einige Details der Vereinbarung. REF: that is, the members of congress have to complete some details of the agreement before they can make the final version of the law public and vote on it. BASE: members of Congress : some details must still negotiate the agreement before they publish the final version of the law and able to vote on it. NEW: members of Congress must negotiate before they publish the final version of the law and must still vote on some details of the agreement.

14 Conclusions & TODOs Pre-ordering with SMT-like system always outperforms baseline MLE, but gains are small Evaluation issue: reference reorderings are very noisy! When input is pre-ordered BLEU improves but KRS decreases... more error analysis needed! Possible reason: the SMT system must be re-trained (or at least tuned) on pre-ordered data More thresholds for rule selection should be tested … other suggestions?

15 Thanks for your attention!


Herunterladen ppt "PRE-ORDERING DEPENDENCY SUBTREES FOR PHRASE-BASED SMT Intern: Arianna Bisazza. Mentors: Alex Ceausu, John Tinsley."

Ähnliche Präsentationen


Google-Anzeigen