Combine the lemon slices, lemon juice, chilies, bay leaf and black pepper into a pint wide mouth canning jar or other glass container with a lid. Cover with the remaining olive oil. Marinate at least 12 hours. I like two or three days. Watch the cheese carefully for signs of mold. Stores in the fridge for up to five days.