Jennifer Bonine: All right, we are back with our last two virtual interviews for those of you watching before we close out for the day with the lightning keynote. I am here right now with Liana Gevorgyan, and Liana, you work for InfoStretch Corporation. For those out there that aren't as familiar with you and your background, could you maybe just give a little history on who you are and what you do and your background and how you came into this profession?

Liana Gevorgyan: Sure. I graduated with a master’s in computer science, and since that time, I'm working in engineering and technology and in quality assurance. It's about fifteen years that I'm in this area, and working for InfoStretch as a senior software manager for the last year.

Jennifer Bonine: Wow, great. And we were talking before you guys joined us just about how this is your first time speaking at this conference.

Liana Gevorgyan: Yeah.

Jennifer Bonine: So maybe for those out there that are sitting in their companies or their homes or wherever they're working, can you give them some idea how you go through the process of actually getting to be a speaker here, and getting this as an opportunity and coming and doing this, so they can kind of understand what that looks like and how you go about doing that?

Liana Gevorgyan: Sure, sure. I'll be happy to answer these. I'm the type of person that I like sharing the ideas, being social, and learning and sharing. I used to be in different European conferences, and in the US, this is a first big conference for me. My director offered it to try, and I was having several ideas, so I picked quality, metrics, and measurements because I feel we're talking a lot about the processes—but let's stress this area, there's not much innovations here, and I think it's important. I picked it out, I sent a proposal to the committee of conference, went to interview, they liked it, so how I'm here today.

Jennifer Bonine: Yeah, great. For those of you out there, one of the nice things is, I think it's an opportunity—if you haven't done it before, if you have an interest in it, if you have something you feel you should be sharing with other people—you know, submit those proposals, get them in, have the opportunity, go through the interview process, and then you can end up here doing conference speaking and being involved in that way as well.

Liana Gevorgyan: Sure, there are so many great people that surrounded me, like, you can just have a chat with them, a few-minute chat. It's very good for sharing and learning. I recommend everyone to follow and don't be shy,—it's not that hard, public speaking. A lot of people I know have this kind of complex ...

Jennifer Bonine: They're afraid of it.

Liana Gevorgyan: But go ahead, and you will become stronger and you will like it.

Jennifer Bonine: Exactly. There you go, give them the courage to try. Let's talk, because your talk that you gave today, everyone out there maybe didn't get to see it, and so can you tell them a little bit about what you were talking about in your session around the metrics and the tracking of information and data, and kind of how you came to that theory of using this methodology to track information and ...

Liana Gevorgyan: Sure.

Jennifer Bonine: ... gather information?

Liana Gevorgyan: I have about an hour presentation about this.

Jennifer Bonine: Boil it down into thirty seconds.

Liana Gevorgyan: Sure. I think this is an important area, and I didn't bring best practices, the processes that everybody’s talking about. I brought the unique way of measuring the quality specific for your company, your product, your process, because you need to have a clear goal—what is your goal for what you’re delivering, and how to make sure that you're delivering a quality one. It's nothing with best practices, but it's the way of measuring that's working for your delivery, and I think this is more up to date, because everybody wants to see it in practice—like my topic, or metrics and measurement in practice—not in theory.

Jennifer Bonine: Right, so actually using it and how to actually use it. Some of the things you talked about in your session as we talked to about, you don't have to buy a tool to do it, you don't have to have a lot of stuff to get started. People have access if they want to go out and see your presentation you did, and take some of those tools and start using them in practice, in your everyday QA management lives to help measure what's going on, and I know we talked about how there's kind of three or four guiding principles around why have metrics and why do that, so maybe share with them just kind of your theory on why do we want to do this.

Liana Gevorgyan: Yeah, I'm not stressing the tools because just changing the tools will not solve the problem. We need to think about how the tools that we're using can work for us. I'm focusing on the data that you're creating based on the process that you're doing … just try, collect, use the historical data to do analysis. I'm hoping that priority of metrics to the graph theory and bringing this to life as new priorities for your project. \

What I'm actually suggesting is to just try to find what are the right metrics for your process, and the best ways analysis helps you when you can find a set. The idea, if I can say in a few sentences, is to define metrics and measurements. If you have metrics and measurements, you should understand. If you understand, you can control, and only when you can control you can keep improving—that is actually happening everywhere.

Jennifer Bonine: Yeah, exactly.

Liana Gevorgyan: You cannot innovate without improving, right?

Jennifer Bonine: Yeah and understanding what you're trying to do. I would say in terms of people out there that are struggling with what their goal is because you know how you say you have to have a goal, so they're struggling with “What is my goal and what should my goal be?” Do you have some strategies or techniques you use for defining the goal of what you're trying to do?

Liana Gevorgyan: I mean, a goal is something that should not be defined by just the QA person ...

Jennifer Bonine: Right.

Liana Gevorgyan: This is organization-level decision.

Jennifer Bonine: Right.

Liana Gevorgyan: It's not going to be clear for every employee, so what I'm doing, the strategy, and the goal for an organization needs to be clear for each employee. It needs to be compounded every month and needs to be shared the first day when you enter into any new company so you can have a clear idea of what you're doing and why you're doing it, what's your goal. Based on your goal, you can make your processes working, you can understand what's important and where you need to put your focus, so I was saying that understanding the goal is important.

In one company, it can be having complete products, for the other it can be being fast in the market. A lot of companies just sharing their products with a better version they don't think of how many issues there are, but they want to hear the feedback and they want to be the first, right? This is a totally different goal and if you keep striking your issues away it will delay it. If it's clear you can understand which direction you need to focus and they're totally different actions, you need to do in the process to succeed best in your goal.

Jennifer Bonine: Mm-hmm (affirmative) and I think, too, some companies like you said, it should be clear to everyone, but I think some of the folks out there may come from organizations where it's not clear or people can't even clearly define it in the top levels of their organization, right? Which makes it hard to funnel that down into their teams and then how to translate maybe that top-level goal into one that makes sense for their team and what they're driving towards. Any advice or recommendations for people in organizations where they're struggling to get that information on how do I drive the goal and what should that goal be?

Liana Gevorgyan: I understand your point, not all the companies have that well-defined goal and strategy, but still, it’s never too late to do this and define it. So in this case, I'd suggest using data-driven approaches so whatever they have, they can analyze the current situation and see what is bringing more value, what is more important for them just try to design goals around this area and at least share this with the team.

Jennifer Bonine: Get consensus around ”Is that right?” after you look at the data.

Liana Gevorgyan: It's not that always, you know, but you always can do analysis and understand … okay, if I'm going to go in this direction, I'm getting this, if I focus more I'll probably have a chance to get this, so let's focus on this side and it still becomes your goal.

Jennifer Bonine: Now, we talked about the metrics and using those and putting them in place. What have you seen the results be for the companies that actually do this? If they use this methodology, what kind of results can you expect by using your framework?

Liana Gevorgyan: I brought some stats for example that I use by each we're having one critical per month customer escalation sum, talking about customer escalation, the goal for us is no critical, no high backs from customers, just lower them down and go with the mediums only. The time that I started applying this model, there were about two, three criticals per month and about six to seven high escalated per month and in six months we cut it critical escalations to zero.

Jennifer Bonine: Wow.

Liana Gevorgyan: And there was no critical escalation for six months, and it's a huge improvement, and we're getting one or two highs per month when we were having six to seven.

Jennifer Bonine: Wow.

Liana Gevorgyan: And you're cutting the patching, like all the switching of the process, we've changed the priority metrics, we added new metrics and how that's worked for us.

Jennifer Bonine: Yeah, and we're using kind of the visibility to people at what you are measuring, what you are looking for from them, you know, and driving that everyone towards the same goal.

Liana Gevorgyan: Sure, sure.

Jennifer Bonine: And knowing what they need to do. What do you see as some of the trends in your job, or what are people worried about, or what are they thinking about in terms of big pain points for you or others?

Liana Gevorgyan: Technology is moving. It's never going to have a best practice, best practices for today, because it's almost late tomorrow, so it's not the best practice for tomorrow. Things keep moving so the clock issue is a part of it, keeping moving, but still during this move you need to be able to face the challenges and solve them, to produce the quality, why we're all here, right? All the QA people, you never see so many QA people under one roof ...

Jennifer Bonine: At the same place, right? Exactly, this is the place to come if you want to network with other fellow QA people. I know you haven't been here real long and you just got in and you're planning to attend what they call the leadership summit on Friday.

Liana Gevorgyan: Sure.

Jennifer Bonine: And for those not able to attend or not coming, what was your motivation for wanting to be a part of that summit that's actually taking place here on Friday?

Liana Gevorgyan: As I told, like a lot of learning and sharing the knowledge. I think this is a very good opportunity to meet with the great leaders in an area, share ideas, get their feedback, and hear back from them. Just a good chance to meet all those great people that we're reading in books ...

Jennifer Bonine: Right. Yeah it is, it's like I always say, coming to these conferences is like a meeting of the mythical creatures in your industry, right? People you hear about that write books that you know, you see their blogs, you see all kinds of stuff, but they're here and they're all here and they're willing to talk to people and give you information and spend time with you. So you know, if a lot of you are following different folks on the blogs and stuff when you look, a lot of them are here and they're willing to sit down and have conversations about what they're thinking or where their heads are at and it's a good place to bounce ideas off of.

I think for those of you that are out there in virtual land, for you guys, there's a forum during some of these sessions to interact and engage so while you maybe can't be here, you can still interact, engage, and talk about what you're taking away from this. Any advice you would give to those folks out there that are watching or things you would tell them about what you've learned in your career in QA so far?

Liana Gevorgyan: Do not be afraid of experimenting, but keep process going and do not break delivery.

Jennifer Bonine: There you go, do not break delivery. Right?

Liana Gevorgyan: Yeah, it's still things that need to go in, but keep sharing the ideas because this is the way that you may thrive and make them a reality. If you're afraid sharing and keep doing best practices, never new ones will come, but you are owners to make new best practices and keep working.

Jennifer Bonine: Keep trying new things. In terms of getting a hold of you, I may not have asked some of the questions that other people wanted to hear from you or they may want more information about your methodology around metrics and how to measure. What's the best way to get a hold of you or get the information from your presentation?

Liana Gevorgyan: In the conference slide, there is a short description of myself and there is a LinkedIn contact, so I'll suggest people to contact me via LinkedIn. I'm going to post the presentation over there in case they want to double check or find some interesting stuff.

Jennifer Bonine: Perfect.

Liana Gevorgyan: Feel free to contact me, I'm open and happy to cover all your questions.

Jennifer Bonine: Great.

Liana Gevorgyan: That's it.

Jennifer Bonine: Well, we are out of time, but I appreciate your time and thank you for being here with us and it’s been a pleasure.

Liana Gevorgyan: Thank you very much for having me here.

Jennifer Bonine: Thank you.

A senior software QA manager at InfoStretch Corporation, Liana Gevorgyan has more than twelve years of experience planning, executing, and delivering on-time quality end-to-end solutions in software development in roles of QA engineer, business analyst, technical project manager, and QA manager. Using weight-based analysis around QA metrics and measurements,  Liana has significantly improved test coverage, test efficiency, and customer satisfaction for about thirty projects and releases including big data, stock market, financial, SAAS based, enterprise, and mobile applications. See Liana’s full profile at LinkedIn.