Baku, April 13, AZERTAC “The State Examination Center (SEC) has excellently organized the English language assessment process,” said a famous English language specialist from the US Nathan Kar in his interview with AZERTAC’s correspondent. -What is the purpose of your visit to Azerbaijan?  “I am here on an English Language Specialist project organized by the US State of Department to assist the State Examination Centre (SEC) staff. The first of three scheduled visits for now helping with try to find out more about the testing process here what SEC does especially in the English language examinations and then helping do some training especially through item writers, markers and teachers and then consultancy trying to build capacity within the SEC so that they can improve their tests, advise them on how to link  them to the Common European Framework Reference for Languages and help them to achieve an international recognition. And this in turn will hopefully lead to some changes in how English gets taught so that the students will get prepared to teach, better how to communicate in English and do well on those tests. And then if they can communicate better then they will be ready to study abroad or do business in the international scene here or overseas.” -How does English Language Specialist project impact the SEC’s work in language assessment? Well, it should impact in several ways. First, in terms of helping build capacity for creating tests that are more communicative, that will be aligned with the Common European Framework of Reference for Languages programme that will assess the ability to use a language not just to memorize lists of vocabulary and no grammar rules, but to be able to use grammar rules, to speak, to write, to read, to listen. This is a problem in a lot of countries not only in Azerbaijan. The US often has similar problems with foreign language teaching especially at the high school level. Also helping to train whatever you decide to do in the end, to score writing and speaking assessment those will introduce in few years. Also helping to build capacity and improve the quality of the testing, training teachers, trainers, item writers and e-markers markers.   -         Since you have been introduced to the testing culture particularly English language testing culturein Azerbaijan, how do you compare the current testing of the SEC with the international testing practices? That answer goes into several different dimensions in no particular order. I was pleasantly impressed with the part of the item writing that goes into the tests, they get revised in high security set up for the tests as administered this preschool event that I was helping with. And especially considering that everything was written and edited previously by non-native English-speakers it was very high quality. Even something written by native speaker needs editing. So it is item writers training and item revision process seems to be doing quite nicely, the psychometrics part I think it is the research department of the SEC. I spent a lot of time within yesterday talking about things. They seem to have; it seems to test very high technical quality in terms of their statistical properties. They go into a lot of detail, they analyze a lot of things, they seem to be doing extremely professional job. That is definitely worth given some credit for. Especially in a country the size of Azerbaijan, I was really surprised to have the resources to do something that proficient with that many guys on the staff and women. Comparing to international standards currently the English tests for school even seem to be a bit short as long as they are only part of decision that is one thing. But if you want an English test that will show how well someone communicate, give separate scores for reading vs listening vs speaking vs writing. It is necessarythat it has to be longer if you want that kind of information and that kind of information is really useful to students, their parents, people who might give them jobs, who want them to be able to speak English or some other language that you are testing. And it is important for teachers too I think, and perhaps for universities if they need someone whocan write, speak better depending on the major. There is a talk of moving English into, having a sort of vocation exam for different levels in the Common European Framework and that necessarily will be longer. Because I don’t know the constraints here. It might not be possible to have a longer test as part of the school test in 9th or 11th grade. But if you want that extra information then you have to have a link and I think some of the test types to be brought into more conformity with the Common European Framework we need a little revision, may be expansion of the types of things that are on there.   -         You had trainings with the SEC staff members, as well as teachers. How were these trainings perceived? No one told me they were unhappy. I was surprised how well things seemed to be going. I sometimes get complaints back home from my graduate students that I lecture too fast and some of them thanked me on Friday for not talking too fast. I thought everyone seemed to be following keeping up even though some of the things I was explaining they had not studied before. I didn’t see any looks of incomprehension, they were participating actively whom we had small group things. I tried to do as much as small group pair works I could. They seem to be doing quite well, they seemed excited and interested I think it is almost a great sign when the people who need to make changes, want to make changes and are not going to be forced to make the changes but they are welcoming the chance to improve things and do a better job. And so I think this is going to be photo ground for sawing seeds to build more capacity here help Azerbaijan make better tests and that in turn will push students to learning, learn to communicate English, push the teachers to teach them to communicate. And that will be a process that takes years. But you have to start somewhere if you want improvement.   -         What are the plans and expectations for the next phases of the project? The October visit will actually begin in September because for the conference I am going to come a little bit earlier and I will do a post-conference workshop. It should be oriented towards teacher training for writing test questions, just test questions in general whether it is for SEC or for their own classes. And then most of October will be teacher training If I recall the schedule I think we have about two days each for writing, reading, listening and speaking. There will be a day or two for grammar and vocabulary. I am sure there will be some consultations also with a SEC staff in there. Then in January it is going to be a primarily consultation with SEC. Some of the details are still not quite set but I am sure there will probably be a little training or something in there that is how things always are. The consultation projects are always lined up adding up a little training, training projects always include a little consultation.        -         We would like to know your opinion on whether the English language subject tests used by SEC exams is satisfactory or not. How do you evaluate that? The process for marking.  I had the process explained to me and it seems to be extremely rigorous the procedure for first is the scoring key and then correcting the answers for credit or partial credit. And then the way that expanded using students sample from their tests and then that is viewed until panel of experts is in agreement on what constitutes acceptable and partially acceptable answers. And this is all for very short question answers with one words, three or five words. Most of them one to three words I think. It seems very rigorously done and likely it should be leading to extremely consistent scores. I understand that the challenge with the success rate on appealing scores, when students run happy, is very low which tells me there is not a lot of error going on in fact that there is a tiny amount of area in the process. And that is achievable. It is not strange that it is so low. But it speaks to extreme care because it is also easy to get this kind of thing wrong and have bad scores and it seems it has been done right here. But you know technical quality programs here are surprisingly high, quite good. I think the challenge is going to be moving on in several years when they start to test writing, speaking, procedures for setting up grading scales or scores then training graders to make sure that they stay trained and consistent and then evaluating how the consistent the scores are. That is going to take some different methodology. But the good news is that I think on IT side the existing infrastructure probably (I am not an IT guy) can be adapted but it is going to be an exciting process moving forward. I am a testing nerd and I think the staff is cool.