Die Präsentation wird geladen. Bitte warten

Die Präsentation wird geladen. Bitte warten

PRE-ORDERING DEPENDENCY SUBTREES FOR PHRASE-BASED SMT Intern: Arianna Bisazza. Mentors: Alex Ceausu, John Tinsley.

Ähnliche Präsentationen


Präsentation zum Thema: "PRE-ORDERING DEPENDENCY SUBTREES FOR PHRASE-BASED SMT Intern: Arianna Bisazza. Mentors: Alex Ceausu, John Tinsley."—  Präsentation transkript:

1 PRE-ORDERING DEPENDENCY SUBTREES FOR PHRASE-BASED SMT Intern: Arianna Bisazza. Mentors: Alex Ceausu, John Tinsley

2 Dependency subtree pre-ordering What if… we cant/dont want to change the decoding process we have dependency parses available …one way to go: pre-order input parse trees, then translate normally Main research problems: how to pre-order? (ordering model) and what? (rule selection)

3 Dependency subtree pre-ordering Die Budapester Staat anwaltschaft hat ihre Ermittlungen zum Vorfall eingeleitet. the Budapest Prosecutors Office has its investigation on the accident initiated ihre |PPOSAT zum |APPRART Vorfall |NN hat |VAFIN anwaltschaft |NN Staat |NN Budapester |NN die |ART eingeleitet |VVPP. |$. Ermittlungen |NN NK SB OC PUNC OA NK MNR NK NN VAFIN VVPP$. NN VAFIN VVPP$. NN VVPP NN VVPP... Permute subtrees (a node + its children) Each subtree processed independently

4 Dependency subtree pre-ordering Die Budapester Staat anwaltschaft hat ihre Ermittlungen zum Vorfall eingeleitet. the Budapest Prosecutors Office has its investigation on the accident initiated ihre |PPOSAT zum |APPRART Vorfall |NN hat |VAFIN anwaltschaft |NN Staat |NN Budapester |NN die |ART eingeleitet |VVPP. |$. Ermittlungen |NN NK SB OC PUNC OA NK MNR NK NN VAFIN VVPP$. NN VAFIN VVPP$. NN VVPP NN VVPP... Permute subtrees (a node + its children) Each subtree processed independently

5 Pre-ordering model (1) – MLE Baseline model: max likelihood MLE (relative frequency-based) Subtree representation: relation type and POS tag OA|NN _OC|VVPP OA|NN _OC|VVPP Prob=0.75 Prob=0.25 OA|NN _OC|VVPP Limitations: - ambiguity due to coarse word classification (only few relation/POS tags) - coverage: many unseen or low-counts subtrees

6 Pre-ordering model (2) – SMT Idea: learn to reorder by SMT! Train a phrase-based system on pairs of original/pre-ordered source language node sequences (subtrees) Advantages: generalization: all node sequences can be processed model flexibility: represent different features as factors tune different model weights by MERT ORIGINAL SB|NN _ROOT|VAFIN OC|VVPP PUNC|$. NK|ART NK|NN NK|NN _SB|NN OA|NN _OC|VVPP... OA|NN _OC|VVPP... ORIGINAL SB|NN _ROOT|VAFIN OC|VVPP PUNC|$. NK|ART NK|NN NK|NN _SB|NN OA|NN _OC|VVPP... OA|NN _OC|VVPP... PRE-ORDERED SB|NN _ROOT|VAFIN OC|VVPP PUNC|$. NK|ART NK|NN NK|NN _SB|NN _OC|VVPP OA|NN... OA|NN _OC|VVPP... PRE-ORDERED SB|NN _ROOT|VAFIN OC|VVPP PUNC|$. NK|ART NK|NN NK|NN _SB|NN _OC|VVPP OA|NN... OA|NN _OC|VVPP...

7 Pre-ordering model (2) – SMT Possible models: original-to-preordered phrase table target (preordered) n-gram language models lexicalized reordering models at the level of relation type, POS tags or words etc. all models log-linearly combined weights tuned by MERT, optimizing reo.score (KRS) ORIGINAL SB|NN| anwaltschaft _ROOT|VAFIN| hat OC|VVPP| eingeleitet PUNC|$.|. NK|ART| die NK|NN| Budapester NK|NN| Staat _SB|NN| anwaltschaft OA|NN| Ermittlungen _OC|VVPP| eingeleitet... ORIGINAL SB|NN| anwaltschaft _ROOT|VAFIN| hat OC|VVPP| eingeleitet PUNC|$.|. NK|ART| die NK|NN| Budapester NK|NN| Staat _SB|NN| anwaltschaft OA|NN| Ermittlungen _OC|VVPP| eingeleitet... Each feature type is represented as a factor, for example:

8 Evaluation Training/dev/test: 495/2.5/2.5K sent. from WMT-12 De-En train data 1.6M/8K/9K training subtrees (rooted at verb nodes) MethodAdd. modelsBLEUKRSACCUNK MLE-rel--57.7771.0146.359.53 MLE-relPOS--55.0071.0345.7524.33 SMT-relPOS (moses) LM(rel) +LM(POS)58.8472.5748.45-- +lexreo(relPOS)60.2473.0549.37-- +lexreo(words)59.8772.9249.05-- +lexreo(nodeSpan)59.7272.9449.03-- MethodAdd. modelsBLEUKRSACCUNK MLE-relPOS--63.3877.2763.0811.91 SMT-relPOS (moses) +lexreo(relPOS)66.7478.2464.69-- 4.8M/23K/24K training subtrees (all with >1 node)

9 Selective pre-ordering Not all subtrees need to be pre-ordered (especially in language pairs like German-English) How to select them? Approach: compute average distortion gain on training data, then only pre-order subtrees with high distortion gain Pre-ordering performances, with two different thresholds Selection%subtreesMethodAdd. modelsBLEUKRSACCUNK None (all subtrees) 100% MLE--63.3877.2763.0811.91 SMT +lexreo(relPOS)66.7478.2464.69-- HRPd15f313% MLE--55.5972.6440.0930.43 SMT +lexreo(relPOS)60.6875.0244.98-- SRPd20f33% MLE--45.1760.9624.348.70 SMT +lexreo(relPOS)51.7065.2128.76--

10 MT experiments Using WMT-12 De-En training and test data InputDL newstest2009newstest2010 BLEUKRSBLEUKRS Original 4 19.4362.7020.9666.00 Reo.all 20.2562.2021.8865.48 Reo.verbRoot 20.2762.2721.9765.48 Reo.HRPd15f3 19.7062.5121.2665.91 Reo.SRPd20f3 19.5562.6721.0865.95 Original 8 20.1863.1421.6866.15 Reo.all 20.3461.8521.9765.00 Reo.verbRoot 20.3862.0022.0965.06 Reo.HRPd15f3 20.3562.6721.8265.89 Reo.SRPd20f3 20.2562.9921.7366.03

11 MT output examples (1) ORI: nach dem steilen Abfall am Morgen konnte die Prager Börse die Verluste korrigieren. REO: nach dem steilen Abfall am Morgen die Prager Börse konnte die Verluste korrigieren. REF: after a sharp drop in the morning, the Prague Stock Market corrected its losses. BASE: after the sharp falls on the morning, the Prague Stock Exchange to correct the losses. NEW: after the sharp falls on the morning the Prague Stock Exchange was able to correct the losses.

12 MT output examples (2) ORI: … über einen Plan, der funktionieren wird und der auf dem Markt auch wirksam sein muss. REO: … über einen Plan, der wird funktionieren und der muss sein auch wirksam auf dem Markt. REF: … on a plan which will function and which also must be effective on the market. BASE: … on a plan that will work and on the market also needs to be effective. NEW: … on a plan that will work and must also be effective on the market.

13 MT output examples (3) ORI: die Kongress Abgeordneten müssen nämlich noch einige Details der Vereinbarung aushandeln, ehe sie die Endfassung des Gesetzes veröffentlichen und darüber abstimmen dürfen. REO: die Kongress Abgeordneten müssen nämlich aushandeln, ehe sie veröffentlichen die Endfassung des Gesetzes und dürfen darüber abstimmen noch einige Details der Vereinbarung. REF: that is, the members of congress have to complete some details of the agreement before they can make the final version of the law public and vote on it. BASE: members of Congress : some details must still negotiate the agreement before they publish the final version of the law and able to vote on it. NEW: members of Congress must negotiate before they publish the final version of the law and must still vote on some details of the agreement.

14 Conclusions & TODOs Pre-ordering with SMT-like system always outperforms baseline MLE, but gains are small Evaluation issue: reference reorderings are very noisy! When input is pre-ordered BLEU improves but KRS decreases... more error analysis needed! Possible reason: the SMT system must be re-trained (or at least tuned) on pre-ordered data More thresholds for rule selection should be tested … other suggestions?

15 Thanks for your attention!


Herunterladen ppt "PRE-ORDERING DEPENDENCY SUBTREES FOR PHRASE-BASED SMT Intern: Arianna Bisazza. Mentors: Alex Ceausu, John Tinsley."

Ähnliche Präsentationen


Google-Anzeigen